[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786292417722232-21710-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 16:20:17 dist-test-slave-s11t krb5kdc[21720](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 16:20:17 dist-test-slave-s11t krb5kdc[21720](info): set up 2 sockets
Aug 09 16:20:17 dist-test-slave-s11t krb5kdc[21720](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260809 16:20:18.998375 21710 mini_kdc.cc:121] Time spent starting KDC: real 1.256s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786292417722232-21710-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786292417722232-21710-0/krb5kdc/test-user.keytab.
Aug 09 16:20:19 dist-test-slave-s11t krb5kdc[21720](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292419, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T16:20:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T16:20:19Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.51.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.51.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.51.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.190.keytab.
Entry for principal kudu/127.21.51.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.190.keytab.
I20260809 16:20:19.120836 21710 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrNqUCn/build/release/bin/kudu
/tmp/dist-test-taskrNqUCn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786292417722232-21710-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786292417722232-21710-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786292417722232-21710-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786292417722232-21710-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.51.190:41443
--webserver_interface=127.21.51.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.190.keytab
--principal=kudu/127.21.51.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.51.148:34653
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.51.190:41443
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786292417722232-21710-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786292417722232-21710-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786292417722232-21710-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:20:19.203549 21741 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:20:19.203783 21741 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:20:19.203820 21741 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:20:19.204977 21741 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 16:20:19.205044 21741 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:20:19.205068 21741 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:20:19.205089 21741 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 16:20:19.205119 21741 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 16:20:19.206586 21741 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.51.148:34653
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786292417722232-21710-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786292417722232-21710-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.51.190:41443
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.190.keytab
--principal=kudu/127.21.51.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.51.190:41443
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786292417722232-21710-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.51.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21741
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786292417722232-21710-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:20:19.206872 21741 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:20:19.207099 21741 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:20:19.212087 21752 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:20:19.212087 21754 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:20:19.212158 21756 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:20:19.212492 21741 server_base.cc:1034] running on GCE node
I20260809 16:20:19.212787 21741 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:20:19.213124 21741 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:20:19.214282 21741 hybrid_clock.cc:648] HybridClock initialized: now 1786292419214274 us; error 111 us; skew 500 ppm
Aug 09 16:20:19 dist-test-slave-s11t krb5kdc[21720](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292419, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.51.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:20:19.216883 21741 init.cc:377] Logged in from keytab as kudu/127.21.51.190@KRBTEST.COM (short username kudu)
I20260809 16:20:19.217353 21741 webserver.cc:460] Webserver started at http://127.21.51.190:40317/ using document root <none> and password file <none>
I20260809 16:20:19.217582 21741 fs_manager.cc:359] Metadata directory not provided
I20260809 16:20:19.217649 21741 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:20:19.217808 21741 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:20:19.218834 21741 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786292417722232-21710-0/minicluster-data/master-0/data/instance:
uuid: "f7db448752c144c389fac2da431e76db"
format_stamp: "Formatted at 2026-08-09 16:20:19 on dist-test-slave-s11t"
server_key: "2923e544d4da948bceff788ab1ff73eb"
server_key_iv: "6dabe58f6d696a696e093c9362a14b44"
server_key_version: "encryptionkey@0"
I20260809 16:20:19.219208 21741 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786292417722232-21710-0/minicluster-data/master-0/wal/instance:
uuid: "f7db448752c144c389fac2da431e76db"
format_stamp: "Formatted at 2026-08-09 16:20:19 on dist-test-slave-s11t"
server_key: "2923e544d4da948bceff788ab1ff73eb"
server_key_iv: "6dabe58f6d696a696e093c9362a14b44"
server_key_version: "encryptionkey@0"
I20260809 16:20:19.220541 21741 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260809 16:20:19.221386 21767 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:20:19.221581 21741 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 16:20:19.221683 21741 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786292417722232-21710-0/minicluster-data/master-0/data,/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786292417722232-21710-0/minicluster-data/master-0/wal
uuid: "f7db448752c144c389fac2da431e76db"
format_stamp: "Formatted at 2026-08-09 16:20:19 on dist-test-slave-s11t"
server_key: "2923e544d4da948bceff788ab1ff73eb"
server_key_iv: "6dabe58f6d696a696e093c9362a14b44"
server_key_version: "encryptionkey@0"
I20260809 16:20:19.221764 21741 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786292417722232-21710-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786292417722232-21710-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786292417722232-21710-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:20:19.247283 21741 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:20:19.249712 21741 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:20:19.249890 21741 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:20:19.254437 21741 rpc_server.cc:307] RPC server started. Bound to: 127.21.51.190:41443
I20260809 16:20:19.254479 21847 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.51.190:41443 every 8 connection(s)
I20260809 16:20:19.254887 21741 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786292417722232-21710-0/minicluster-data/master-0/data/info.pb
I20260809 16:20:19.255599 21710 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrNqUCn/build/release/bin/kudu as pid 21741
I20260809 16:20:19.255712 21710 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786292417722232-21710-0/minicluster-data/master-0/wal/instance
I20260809 16:20:19.255842 21850 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:20:19.256399 21710 external_mini_cluster.cc:1428] Setting key 0309cf6efef0bea1e4d552a09bd559c1
I20260809 16:20:19.259059 21850 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f7db448752c144c389fac2da431e76db: Bootstrap starting.
I20260809 16:20:19.259788 21850 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f7db448752c144c389fac2da431e76db: Neither blocks nor log segments found. Creating new log.
I20260809 16:20:19.260131 21850 log.cc:824] T 00000000000000000000000000000000 P f7db448752c144c389fac2da431e76db: Log is configured to *not* fsync() on all Append() calls
I20260809 16:20:19.260977 21850 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f7db448752c144c389fac2da431e76db: No bootstrap required, opened a new log
I20260809 16:20:19.263587 21850 raft_consensus.cc:348] T 00000000000000000000000000000000 P f7db448752c144c389fac2da431e76db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f7db448752c144c389fac2da431e76db" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 41443 } }
I20260809 16:20:19.263722 21850 raft_consensus.cc:374] T 00000000000000000000000000000000 P f7db448752c144c389fac2da431e76db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:20:19.263746 21850 raft_consensus.cc:729] T 00000000000000000000000000000000 P f7db448752c144c389fac2da431e76db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f7db448752c144c389fac2da431e76db, State: Initialized, Role: FOLLOWER
I20260809 16:20:19.263873 21850 consensus_queue.cc:260] T 00000000000000000000000000000000 P f7db448752c144c389fac2da431e76db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f7db448752c144c389fac2da431e76db" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 41443 } }
I20260809 16:20:19.263949 21850 raft_consensus.cc:388] T 00000000000000000000000000000000 P f7db448752c144c389fac2da431e76db [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 16:20:19.263998 21850 raft_consensus.cc:482] T 00000000000000000000000000000000 P f7db448752c144c389fac2da431e76db [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 16:20:19.264047 21850 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f7db448752c144c389fac2da431e76db [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:20:19.264597 21850 raft_consensus.cc:504] T 00000000000000000000000000000000 P f7db448752c144c389fac2da431e76db [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f7db448752c144c389fac2da431e76db" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 41443 } }
I20260809 16:20:19.264739 21850 leader_election.cc:302] T 00000000000000000000000000000000 P f7db448752c144c389fac2da431e76db [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f7db448752c144c389fac2da431e76db; no voters:
I20260809 16:20:19.264909 21850 leader_election.cc:288] T 00000000000000000000000000000000 P f7db448752c144c389fac2da431e76db [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 16:20:19.264937 21855 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f7db448752c144c389fac2da431e76db [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:20:19.265107 21855 raft_consensus.cc:686] T 00000000000000000000000000000000 P f7db448752c144c389fac2da431e76db [term 1 LEADER]: Becoming Leader. State: Replica: f7db448752c144c389fac2da431e76db, State: Running, Role: LEADER
I20260809 16:20:19.265178 21850 sys_catalog.cc:564] T 00000000000000000000000000000000 P f7db448752c144c389fac2da431e76db [sys.catalog]: configured and running, proceeding with master startup.
I20260809 16:20:19.265202 21855 consensus_queue.cc:237] T 00000000000000000000000000000000 P f7db448752c144c389fac2da431e76db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f7db448752c144c389fac2da431e76db" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 41443 } }
I20260809 16:20:19.265491 21855 sys_catalog.cc:455] T 00000000000000000000000000000000 P f7db448752c144c389fac2da431e76db [sys.catalog]: SysCatalogTable state changed. Reason: New leader f7db448752c144c389fac2da431e76db. Latest consensus state: current_term: 1 leader_uuid: "f7db448752c144c389fac2da431e76db" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f7db448752c144c389fac2da431e76db" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 41443 } } }
I20260809 16:20:19.265587 21855 sys_catalog.cc:458] T 00000000000000000000000000000000 P f7db448752c144c389fac2da431e76db [sys.catalog]: This master's current role is: LEADER
I20260809 16:20:19.265502 21857 sys_catalog.cc:455] T 00000000000000000000000000000000 P f7db448752c144c389fac2da431e76db [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f7db448752c144c389fac2da431e76db" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f7db448752c144c389fac2da431e76db" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 41443 } } }
I20260809 16:20:19.265683 21857 sys_catalog.cc:458] T 00000000000000000000000000000000 P f7db448752c144c389fac2da431e76db [sys.catalog]: This master's current role is: LEADER
I20260809 16:20:19.265858 21863 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 16:20:19.266355 21863 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 16:20:19.267935 21863 catalog_manager.cc:1347] Generated new cluster ID: 53045279daa2442095c557ec30ccc00f
I20260809 16:20:19.267990 21863 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 09 16:20:19 dist-test-slave-s11t krb5kdc[21720](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292419, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.51.190@KRBTEST.COM
I20260809 16:20:19.274112 21852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:19.257467 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48854 (local address 127.21.51.190:41443)
0809 16:20:19.257635 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0809 16:20:19.257642 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 16:20:19.258020 (+ 378us) server_negotiation.cc:407] Connection header received
0809 16:20:19.260975 (+ 2955us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:20:19.260982 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:20:19.261055 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:20:19.261175 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:20:19.263351 (+ 2176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:19.264450 (+ 1099us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:19.266071 (+ 1621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:19.266744 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:19.266871 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:19.270432 (+ 3561us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:20:19.270436 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:20:19.270436 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:20:19.270447 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:20:19.272104 (+ 1657us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:20:19.272560 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:20:19.272563 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:20:19.272564 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:20:19.272655 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:20:19.272958 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:20:19.272959 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:20:19.272959 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:20:19.273190 (+ 231us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:20:19.273362 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0809 16:20:19.273645 (+ 283us) server_negotiation.cc:299] Negotiation successful
0809 16:20:19.273776 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":117,"thread_start_us":84,"threads_started":1}
I20260809 16:20:19.285100 21863 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 16:20:19.285632 21863 catalog_manager.cc:1504] Loading token signing keys...
I20260809 16:20:19.288431 21863 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f7db448752c144c389fac2da431e76db: Generated new TSK 0
I20260809 16:20:19.288599 21863 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.21.51.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.51.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.51.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.129.keytab.
Entry for principal kudu/127.21.51.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.129.keytab.
I20260809 16:20:19.323843 21710 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrNqUCn/build/release/bin/kudu
/tmp/dist-test-taskrNqUCn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786292417722232-21710-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786292417722232-21710-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786292417722232-21710-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786292417722232-21710-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.51.129:0
--local_ip_for_outbound_sockets=127.21.51.129
--webserver_interface=127.21.51.129
--webserver_port=0
--tserver_master_addrs=127.21.51.190:41443
--keytab_file=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.129.keytab
--principal=kudu/127.21.51.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.51.148:34653
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786292417722232-21710-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786292417722232-21710-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786292417722232-21710-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:20:19.414269 21885 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:20:19.414501 21885 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:20:19.414542 21885 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:20:19.415686 21885 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:20:19.415757 21885 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:20:19.415805 21885 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.51.129
I20260809 16:20:19.417342 21885 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.51.148:34653
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786292417722232-21710-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786292417722232-21710-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.129.keytab
--principal=kudu/127.21.51.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.51.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786292417722232-21710-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.51.129
--webserver_port=0
--tserver_master_addrs=127.21.51.190:41443
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21885
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.51.129
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786292417722232-21710-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:20:19.417615 21885 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:20:19.417848 21885 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:20:19.423002 21892 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:20:19.423067 21894 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:20:19.422974 21891 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:20:19.423220 21885 server_base.cc:1034] running on GCE node
I20260809 16:20:19.423470 21885 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:20:19.423715 21885 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:20:19.424875 21885 hybrid_clock.cc:648] HybridClock initialized: now 1786292419424855 us; error 53 us; skew 500 ppm
Aug 09 16:20:19 dist-test-slave-s11t krb5kdc[21720](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292419, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.51.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:20:19.427469 21885 init.cc:377] Logged in from keytab as kudu/127.21.51.129@KRBTEST.COM (short username kudu)
I20260809 16:20:19.427906 21885 webserver.cc:460] Webserver started at http://127.21.51.129:45565/ using document root <none> and password file <none>
I20260809 16:20:19.428143 21885 fs_manager.cc:359] Metadata directory not provided
I20260809 16:20:19.428213 21885 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:20:19.428370 21885 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:20:19.429603 21885 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786292417722232-21710-0/minicluster-data/ts-0/data/instance:
uuid: "e1b346ff492a45b8a8c2cae77c414d79"
format_stamp: "Formatted at 2026-08-09 16:20:19 on dist-test-slave-s11t"
server_key: "448277c5ea43ced2e94252b659bc8e4e"
server_key_iv: "479acca1c4ec78a757ff43ee6dfafe52"
server_key_version: "encryptionkey@0"
I20260809 16:20:19.429982 21885 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786292417722232-21710-0/minicluster-data/ts-0/wal/instance:
uuid: "e1b346ff492a45b8a8c2cae77c414d79"
format_stamp: "Formatted at 2026-08-09 16:20:19 on dist-test-slave-s11t"
server_key: "448277c5ea43ced2e94252b659bc8e4e"
server_key_iv: "479acca1c4ec78a757ff43ee6dfafe52"
server_key_version: "encryptionkey@0"
I20260809 16:20:19.431354 21885 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 16:20:19.432096 21903 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:20:19.432281 21885 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 16:20:19.432401 21885 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786292417722232-21710-0/minicluster-data/ts-0/data,/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786292417722232-21710-0/minicluster-data/ts-0/wal
uuid: "e1b346ff492a45b8a8c2cae77c414d79"
format_stamp: "Formatted at 2026-08-09 16:20:19 on dist-test-slave-s11t"
server_key: "448277c5ea43ced2e94252b659bc8e4e"
server_key_iv: "479acca1c4ec78a757ff43ee6dfafe52"
server_key_version: "encryptionkey@0"
I20260809 16:20:19.432477 21885 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786292417722232-21710-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786292417722232-21710-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786292417722232-21710-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:20:19.454988 21885 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:20:19.457178 21885 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:20:19.457343 21885 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:20:19.457609 21885 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:20:19.457934 21885 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:20:19.457991 21885 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:20:19.458032 21885 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:20:19.458078 21885 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:20:19.464895 21885 rpc_server.cc:307] RPC server started. Bound to: 127.21.51.129:37215
I20260809 16:20:19.465013 22080 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.51.129:37215 every 8 connection(s)
I20260809 16:20:19.465332 21885 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786292417722232-21710-0/minicluster-data/ts-0/data/info.pb
I20260809 16:20:19.469374 21710 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrNqUCn/build/release/bin/kudu as pid 21885
I20260809 16:20:19.469452 21710 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786292417722232-21710-0/minicluster-data/ts-0/wal/instance
I20260809 16:20:19.469614 21710 external_mini_cluster.cc:1428] Setting key 6ea85defc069e4f8c368789c7396a464
Aug 09 16:20:19 dist-test-slave-s11t krb5kdc[21720](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292419, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.51.129@KRBTEST.COM for kudu/127.21.51.190@KRBTEST.COM
I20260809 16:20:19.476586 21852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:19.466175 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.51.129:49775 (local address 127.21.51.190:41443)
0809 16:20:19.466232 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0809 16:20:19.466234 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:20:19.466471 (+ 237us) server_negotiation.cc:407] Connection header received
0809 16:20:19.467094 (+ 623us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:20:19.467095 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:20:19.467108 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:20:19.467176 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:20:19.468418 (+ 1242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:19.468997 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:19.470470 (+ 1473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:19.470734 (+ 264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:19.470823 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:19.473444 (+ 2621us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:20:19.473445 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:20:19.473445 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:20:19.473452 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:20:19.474964 (+ 1512us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:20:19.475535 (+ 571us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:20:19.475536 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:20:19.475537 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:20:19.475589 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:20:19.475903 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:20:19.475904 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:20:19.475904 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:20:19.476005 (+ 101us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:20:19.476055 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0809 16:20:19.476449 (+ 394us) server_negotiation.cc:299] Negotiation successful
0809 16:20:19.476519 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260809 16:20:19.476989 22085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:19.466134 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:41443 (local address 127.21.51.129:49775)
0809 16:20:19.466322 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0809 16:20:19.466360 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0809 16:20:19.466937 (+ 577us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:20:19.467275 (+ 338us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:20:19.467280 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:20:19.468094 (+ 814us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:20:19.468313 (+ 219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:20:19.468318 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:20:19.469087 (+ 769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:20:19.469088 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:20:19.470364 (+ 1276us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:20:19.470368 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:20:19.470830 (+ 462us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:20:19.470832 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:20:19.470983 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:19.471595 (+ 612us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:20:19.471604 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:20:19.473350 (+ 1746us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:20:19.475057 (+ 1707us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:20:19.475065 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:20:19.475067 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:20:19.475450 (+ 383us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:20:19.475659 (+ 209us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:20:19.475660 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:20:19.475661 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:20:19.475821 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:20:19.476078 (+ 257us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:20:19.476086 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:20:19.476293 (+ 207us) client_negotiation.cc:769] Sending connection context
0809 16:20:19.476438 (+ 145us) client_negotiation.cc:240] Negotiation successful
0809 16:20:19.476648 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":100,"threads_started":1}
I20260809 16:20:19.477602 22081 heartbeater.cc:347] Connected to a master server at 127.21.51.190:41443
I20260809 16:20:19.477723 22081 heartbeater.cc:464] Registering TS with master...
I20260809 16:20:19.477938 22081 heartbeater.cc:511] Master 127.21.51.190:41443 requested a full tablet report, sending...
I20260809 16:20:19.478574 21790 ts_manager.cc:194] Registered new tserver with Master: e1b346ff492a45b8a8c2cae77c414d79 (127.21.51.129:37215)
I20260809 16:20:19.479655 21790 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.51.129@KRBTEST.COM'} at 127.21.51.129:49775
WARNING: no policy specified for kudu/127.21.51.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.51.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.51.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.130.keytab.
Entry for principal kudu/127.21.51.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.130.keytab.
I20260809 16:20:19.497583 21710 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrNqUCn/build/release/bin/kudu
/tmp/dist-test-taskrNqUCn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786292417722232-21710-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786292417722232-21710-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786292417722232-21710-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786292417722232-21710-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.51.130:0
--local_ip_for_outbound_sockets=127.21.51.130
--webserver_interface=127.21.51.130
--webserver_port=0
--tserver_master_addrs=127.21.51.190:41443
--keytab_file=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.130.keytab
--principal=kudu/127.21.51.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.51.148:34653
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786292417722232-21710-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786292417722232-21710-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786292417722232-21710-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:20:19.581560 22089 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:20:19.581810 22089 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:20:19.581836 22089 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:20:19.582998 22089 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:20:19.583041 22089 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:20:19.583067 22089 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.51.130
I20260809 16:20:19.584527 22089 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.51.148:34653
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786292417722232-21710-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786292417722232-21710-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.130.keytab
--principal=kudu/127.21.51.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.51.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786292417722232-21710-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.51.130
--webserver_port=0
--tserver_master_addrs=127.21.51.190:41443
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22089
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.51.130
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786292417722232-21710-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:20:19.584790 22089 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:20:19.585029 22089 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:20:19.590170 22096 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:20:19.590159 22099 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:20:19.590173 22095 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:20:19.590334 22089 server_base.cc:1034] running on GCE node
I20260809 16:20:19.590639 22089 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:20:19.590893 22089 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:20:19.592059 22089 hybrid_clock.cc:648] HybridClock initialized: now 1786292419592040 us; error 33 us; skew 500 ppm
Aug 09 16:20:19 dist-test-slave-s11t krb5kdc[21720](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292419, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.51.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:20:19.594691 22089 init.cc:377] Logged in from keytab as kudu/127.21.51.130@KRBTEST.COM (short username kudu)
I20260809 16:20:19.595187 22089 webserver.cc:460] Webserver started at http://127.21.51.130:44495/ using document root <none> and password file <none>
I20260809 16:20:19.595427 22089 fs_manager.cc:359] Metadata directory not provided
I20260809 16:20:19.595498 22089 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:20:19.595652 22089 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:20:19.596742 22089 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786292417722232-21710-0/minicluster-data/ts-1/data/instance:
uuid: "9d6cb28ad94b4b6f8e82f72e0e36eb0e"
format_stamp: "Formatted at 2026-08-09 16:20:19 on dist-test-slave-s11t"
server_key: "16ea55d3683e30facc1c596e3a617e28"
server_key_iv: "8c6240c0f0744ef5f555523b7d421d4d"
server_key_version: "encryptionkey@0"
I20260809 16:20:19.597126 22089 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786292417722232-21710-0/minicluster-data/ts-1/wal/instance:
uuid: "9d6cb28ad94b4b6f8e82f72e0e36eb0e"
format_stamp: "Formatted at 2026-08-09 16:20:19 on dist-test-slave-s11t"
server_key: "16ea55d3683e30facc1c596e3a617e28"
server_key_iv: "8c6240c0f0744ef5f555523b7d421d4d"
server_key_version: "encryptionkey@0"
I20260809 16:20:19.598491 22089 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:20:19.599280 22109 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:20:19.599498 22089 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:20:19.599606 22089 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786292417722232-21710-0/minicluster-data/ts-1/data,/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786292417722232-21710-0/minicluster-data/ts-1/wal
uuid: "9d6cb28ad94b4b6f8e82f72e0e36eb0e"
format_stamp: "Formatted at 2026-08-09 16:20:19 on dist-test-slave-s11t"
server_key: "16ea55d3683e30facc1c596e3a617e28"
server_key_iv: "8c6240c0f0744ef5f555523b7d421d4d"
server_key_version: "encryptionkey@0"
I20260809 16:20:19.599680 22089 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786292417722232-21710-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786292417722232-21710-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786292417722232-21710-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:20:19.623085 22089 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:20:19.625733 22089 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:20:19.625911 22089 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:20:19.626168 22089 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:20:19.626503 22089 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:20:19.626561 22089 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:20:19.626606 22089 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:20:19.626631 22089 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:20:19.633119 22089 rpc_server.cc:307] RPC server started. Bound to: 127.21.51.130:45589
I20260809 16:20:19.633170 22280 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.51.130:45589 every 8 connection(s)
I20260809 16:20:19.633495 22089 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786292417722232-21710-0/minicluster-data/ts-1/data/info.pb
Aug 09 16:20:19 dist-test-slave-s11t krb5kdc[21720](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292419, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.51.130@KRBTEST.COM for kudu/127.21.51.190@KRBTEST.COM
I20260809 16:20:19.642944 22285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:19.634286 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:41443 (local address 127.21.51.130:59809)
0809 16:20:19.634476 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0809 16:20:19.634519 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0809 16:20:19.635024 (+ 505us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:20:19.635282 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:20:19.635286 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:20:19.635899 (+ 613us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:20:19.636099 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:20:19.636104 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:20:19.637001 (+ 897us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:20:19.637002 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:20:19.637802 (+ 800us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:20:19.637804 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:20:19.638152 (+ 348us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:20:19.638153 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:20:19.638258 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:19.638762 (+ 504us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:20:19.638770 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:20:19.640164 (+ 1394us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:20:19.641629 (+ 1465us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:20:19.641635 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:20:19.641636 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:20:19.641910 (+ 274us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:20:19.642097 (+ 187us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:20:19.642098 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:20:19.642098 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:20:19.642197 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:20:19.642466 (+ 269us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:20:19.642470 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:20:19.642579 (+ 109us) client_negotiation.cc:769] Sending connection context
0809 16:20:19.642659 (+ 80us) client_negotiation.cc:240] Negotiation successful
0809 16:20:19.642786 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":107,"threads_started":1}
I20260809 16:20:19.642964 21852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:19.634304 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.51.130:59809 (local address 127.21.51.190:41443)
0809 16:20:19.634371 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0809 16:20:19.634373 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:20:19.634611 (+ 238us) server_negotiation.cc:407] Connection header received
0809 16:20:19.635108 (+ 497us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:20:19.635109 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:20:19.635119 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:20:19.635223 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:20:19.636196 (+ 973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:19.636915 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:19.637885 (+ 970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:19.638075 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:19.638132 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:19.640262 (+ 2130us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:20:19.640263 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:20:19.640263 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:20:19.640270 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:20:19.641559 (+ 1289us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:20:19.641979 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:20:19.641980 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:20:19.641980 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:20:19.642036 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:20:19.642278 (+ 242us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:20:19.642279 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:20:19.642279 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:20:19.642399 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:20:19.642466 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0809 16:20:19.642716 (+ 250us) server_negotiation.cc:299] Negotiation successful
0809 16:20:19.642797 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260809 16:20:19.643025 21710 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrNqUCn/build/release/bin/kudu as pid 22089
I20260809 16:20:19.643116 21710 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786292417722232-21710-0/minicluster-data/ts-1/wal/instance
I20260809 16:20:19.643266 21710 external_mini_cluster.cc:1428] Setting key 3cc07ff942141ad0e6367344104b5402
I20260809 16:20:19.643954 22281 heartbeater.cc:347] Connected to a master server at 127.21.51.190:41443
I20260809 16:20:19.644066 22281 heartbeater.cc:464] Registering TS with master...
I20260809 16:20:19.644243 22281 heartbeater.cc:511] Master 127.21.51.190:41443 requested a full tablet report, sending...
I20260809 16:20:19.644646 21790 ts_manager.cc:194] Registered new tserver with Master: 9d6cb28ad94b4b6f8e82f72e0e36eb0e (127.21.51.130:45589)
I20260809 16:20:19.645056 21790 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.51.130@KRBTEST.COM'} at 127.21.51.130:59809
WARNING: no policy specified for kudu/127.21.51.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.51.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.51.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.131.keytab.
Entry for principal kudu/127.21.51.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.131.keytab.
I20260809 16:20:19.669173 21710 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrNqUCn/build/release/bin/kudu
/tmp/dist-test-taskrNqUCn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786292417722232-21710-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786292417722232-21710-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786292417722232-21710-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786292417722232-21710-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.51.131:0
--local_ip_for_outbound_sockets=127.21.51.131
--webserver_interface=127.21.51.131
--webserver_port=0
--tserver_master_addrs=127.21.51.190:41443
--keytab_file=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.131.keytab
--principal=kudu/127.21.51.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.51.148:34653
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786292417722232-21710-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786292417722232-21710-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786292417722232-21710-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:20:19.751341 22291 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:20:19.751576 22291 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:20:19.751616 22291 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:20:19.752781 22291 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:20:19.752851 22291 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:20:19.752902 22291 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.51.131
I20260809 16:20:19.754364 22291 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.51.148:34653
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786292417722232-21710-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786292417722232-21710-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.131.keytab
--principal=kudu/127.21.51.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.51.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786292417722232-21710-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.51.131
--webserver_port=0
--tserver_master_addrs=127.21.51.190:41443
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22291
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.51.131
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786292417722232-21710-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:20:19.754659 22291 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:20:19.754896 22291 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:20:19.760023 22301 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:20:19.760017 22297 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:20:19.760054 22291 server_base.cc:1034] running on GCE node
W20260809 16:20:19.760015 22299 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:20:19.760522 22291 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:20:19.760792 22291 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:20:19.761965 22291 hybrid_clock.cc:648] HybridClock initialized: now 1786292419761957 us; error 43 us; skew 500 ppm
Aug 09 16:20:19 dist-test-slave-s11t krb5kdc[21720](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292419, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.51.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:20:19.764845 22291 init.cc:377] Logged in from keytab as kudu/127.21.51.131@KRBTEST.COM (short username kudu)
I20260809 16:20:19.765338 22291 webserver.cc:460] Webserver started at http://127.21.51.131:45037/ using document root <none> and password file <none>
I20260809 16:20:19.765594 22291 fs_manager.cc:359] Metadata directory not provided
I20260809 16:20:19.765664 22291 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:20:19.765843 22291 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:20:19.767014 22291 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786292417722232-21710-0/minicluster-data/ts-2/data/instance:
uuid: "d4adea7c9aaf49aaa514a93085a756a2"
format_stamp: "Formatted at 2026-08-09 16:20:19 on dist-test-slave-s11t"
server_key: "9f3bee30ae237225c35a4ec714d69276"
server_key_iv: "1f6e5f042f3d74c56bd4d1c92edc65cb"
server_key_version: "encryptionkey@0"
I20260809 16:20:19.767370 22291 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786292417722232-21710-0/minicluster-data/ts-2/wal/instance:
uuid: "d4adea7c9aaf49aaa514a93085a756a2"
format_stamp: "Formatted at 2026-08-09 16:20:19 on dist-test-slave-s11t"
server_key: "9f3bee30ae237225c35a4ec714d69276"
server_key_iv: "1f6e5f042f3d74c56bd4d1c92edc65cb"
server_key_version: "encryptionkey@0"
I20260809 16:20:19.768805 22291 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 16:20:19.769583 22313 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:20:19.769814 22291 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:20:19.769908 22291 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786292417722232-21710-0/minicluster-data/ts-2/data,/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786292417722232-21710-0/minicluster-data/ts-2/wal
uuid: "d4adea7c9aaf49aaa514a93085a756a2"
format_stamp: "Formatted at 2026-08-09 16:20:19 on dist-test-slave-s11t"
server_key: "9f3bee30ae237225c35a4ec714d69276"
server_key_iv: "1f6e5f042f3d74c56bd4d1c92edc65cb"
server_key_version: "encryptionkey@0"
I20260809 16:20:19.769984 22291 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786292417722232-21710-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786292417722232-21710-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786292417722232-21710-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:20:19.783710 22291 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:20:19.785593 22291 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:20:19.785753 22291 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:20:19.785979 22291 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:20:19.786290 22291 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:20:19.786324 22291 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:20:19.786365 22291 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:20:19.786402 22291 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:20:19.792632 22291 rpc_server.cc:307] RPC server started. Bound to: 127.21.51.131:42107
I20260809 16:20:19.792694 22487 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.51.131:42107 every 8 connection(s)
I20260809 16:20:19.793013 22291 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786292417722232-21710-0/minicluster-data/ts-2/data/info.pb
I20260809 16:20:19.793853 21710 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrNqUCn/build/release/bin/kudu as pid 22291
I20260809 16:20:19.793953 21710 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786292417722232-21710-0/minicluster-data/ts-2/wal/instance
I20260809 16:20:19.794122 21710 external_mini_cluster.cc:1428] Setting key b511c41a8409580fe97064ed3efcb85c
Aug 09 16:20:19 dist-test-slave-s11t krb5kdc[21720](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292419, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.51.131@KRBTEST.COM for kudu/127.21.51.190@KRBTEST.COM
I20260809 16:20:19.802634 21852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:19.794090 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.51.131:56423 (local address 127.21.51.190:41443)
0809 16:20:19.794134 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0809 16:20:19.794136 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:20:19.794147 (+ 11us) server_negotiation.cc:407] Connection header received
0809 16:20:19.794780 (+ 633us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:20:19.794782 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:20:19.794794 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:20:19.794848 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:20:19.795936 (+ 1088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:19.796448 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:19.797526 (+ 1078us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:19.797744 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:19.797830 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:19.799932 (+ 2102us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:20:19.799933 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:20:19.799934 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:20:19.799941 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:20:19.801191 (+ 1250us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:20:19.801613 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:20:19.801614 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:20:19.801615 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:20:19.801675 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:20:19.801928 (+ 253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:20:19.801929 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:20:19.801929 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:20:19.802047 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:20:19.802125 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0809 16:20:19.802459 (+ 334us) server_negotiation.cc:299] Negotiation successful
0809 16:20:19.802541 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260809 16:20:19.802836 22491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:19.793809 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:41443 (local address 127.21.51.131:56423)
0809 16:20:19.794006 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0809 16:20:19.794049 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0809 16:20:19.794689 (+ 640us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:20:19.794921 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:20:19.794925 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:20:19.795626 (+ 701us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:20:19.795821 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:20:19.795826 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:20:19.796545 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:20:19.796546 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:20:19.797433 (+ 887us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:20:19.797436 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:20:19.797843 (+ 407us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:20:19.797844 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:20:19.797980 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:19.798436 (+ 456us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:20:19.798445 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:20:19.799822 (+ 1377us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:20:19.801277 (+ 1455us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:20:19.801284 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:20:19.801286 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:20:19.801519 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:20:19.801749 (+ 230us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:20:19.801750 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:20:19.801750 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:20:19.801861 (+ 111us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:20:19.802115 (+ 254us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:20:19.802122 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:20:19.802260 (+ 138us) client_negotiation.cc:769] Sending connection context
0809 16:20:19.802432 (+ 172us) client_negotiation.cc:240] Negotiation successful
0809 16:20:19.802592 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":117,"threads_started":1}
I20260809 16:20:19.803303 22488 heartbeater.cc:347] Connected to a master server at 127.21.51.190:41443
I20260809 16:20:19.803414 22488 heartbeater.cc:464] Registering TS with master...
I20260809 16:20:19.803625 22488 heartbeater.cc:511] Master 127.21.51.190:41443 requested a full tablet report, sending...
I20260809 16:20:19.803984 21790 ts_manager.cc:194] Registered new tserver with Master: d4adea7c9aaf49aaa514a93085a756a2 (127.21.51.131:42107)
I20260809 16:20:19.804350 21790 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.51.131@KRBTEST.COM'} at 127.21.51.131:56423
I20260809 16:20:19.806274 21710 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 09 16:20:19 dist-test-slave-s11t krb5kdc[21720](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292419, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 09 16:20:19 dist-test-slave-s11t krb5kdc[21720](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292419, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.21.51.129@KRBTEST.COM
I20260809 16:20:19.829944 22496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:19.822135 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59944 (local address 127.21.51.129:37215)
0809 16:20:19.822312 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0809 16:20:19.822316 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 16:20:19.822327 (+ 11us) server_negotiation.cc:407] Connection header received
0809 16:20:19.822352 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:20:19.822353 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:20:19.822367 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:20:19.822435 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:20:19.822965 (+ 530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:19.823714 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:19.824413 (+ 699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:19.824777 (+ 364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:19.824834 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:19.827054 (+ 2220us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:20:19.827062 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:20:19.827062 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:20:19.827069 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:20:19.828944 (+ 1875us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:20:19.829269 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:20:19.829272 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:20:19.829272 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:20:19.829328 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:20:19.829500 (+ 172us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:20:19.829501 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:20:19.829501 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:20:19.829626 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:20:19.829725 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0809 16:20:19.829791 (+ 66us) server_negotiation.cc:299] Negotiation successful
0809 16:20:19.829865 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":136,"thread_start_us":102,"threads_started":1}
W20260809 16:20:19.830322 21957 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:59944
Aug 09 16:20:19 dist-test-slave-s11t krb5kdc[21720](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292419, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 09 16:20:19 dist-test-slave-s11t krb5kdc[21720](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292419, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.51.129@KRBTEST.COM
I20260809 16:20:19.852795 22496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:19.846063 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59954 (local address 127.21.51.129:37215)
0809 16:20:19.846149 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0809 16:20:19.846151 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:20:19.846199 (+ 48us) server_negotiation.cc:407] Connection header received
0809 16:20:19.846273 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:20:19.846274 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:20:19.846285 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:20:19.846335 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:20:19.846866 (+ 531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:19.847345 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:19.848031 (+ 686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:19.848221 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:19.848272 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:19.850384 (+ 2112us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:20:19.850385 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:20:19.850385 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:20:19.850392 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:20:19.851775 (+ 1383us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:20:19.852061 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:20:19.852062 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:20:19.852062 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:20:19.852110 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:20:19.852251 (+ 141us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:20:19.852251 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:20:19.852252 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:20:19.852329 (+ 77us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:20:19.852414 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0809 16:20:19.852579 (+ 165us) server_negotiation.cc:299] Negotiation successful
0809 16:20:19.852638 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260809 16:20:19.853546 21710 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrNqUCn/build/release/bin/kudu with pid 21885
I20260809 16:20:19.868422 21710 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrNqUCn/build/release/bin/kudu with pid 22089
I20260809 16:20:19.887233 21710 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrNqUCn/build/release/bin/kudu with pid 22291
I20260809 16:20:19.898041 21710 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrNqUCn/build/release/bin/kudu with pid 21741
2026-08-09T16:20:19Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (2187 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786292417722232-21710-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 16:20:19 dist-test-slave-s11t krb5kdc[22504](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 16:20:19 dist-test-slave-s11t krb5kdc[22504](info): set up 2 sockets
Aug 09 16:20:19 dist-test-slave-s11t krb5kdc[22504](info): commencing operation
krb5kdc: starting...
W20260809 16:20:21.135764 21710 mini_kdc.cc:121] Time spent starting KDC: real 1.212s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786292417722232-21710-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786292417722232-21710-0/krb5kdc/test-user.keytab.
Aug 09 16:20:21 dist-test-slave-s11t krb5kdc[22504](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292421, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T16:20:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T16:20:21Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.51.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.51.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.51.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.190.keytab.
Entry for principal kudu/127.21.51.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.190.keytab.
I20260809 16:20:21.261710 21710 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrNqUCn/build/release/bin/kudu
/tmp/dist-test-taskrNqUCn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786292417722232-21710-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786292417722232-21710-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786292417722232-21710-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786292417722232-21710-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.51.190:45243
--webserver_interface=127.21.51.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.190.keytab
--principal=kudu/127.21.51.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.51.148:41743
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.51.190:45243
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786292417722232-21710-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786292417722232-21710-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786292417722232-21710-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:20:21.343379 22521 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:20:21.343575 22521 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:20:21.343596 22521 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:20:21.344766 22521 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 16:20:21.344810 22521 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:20:21.344825 22521 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:20:21.344836 22521 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 16:20:21.344902 22521 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 16:20:21.346345 22521 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.51.148:41743
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786292417722232-21710-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786292417722232-21710-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.51.190:45243
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.190.keytab
--principal=kudu/127.21.51.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.51.190:45243
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786292417722232-21710-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.51.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22521
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786292417722232-21710-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:20:21.346611 22521 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:20:21.346838 22521 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:20:21.351701 22539 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:20:21.351718 22532 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:20:21.351747 22537 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:20:21.351912 22521 server_base.cc:1034] running on GCE node
I20260809 16:20:21.352208 22521 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:20:21.352535 22521 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:20:21.353693 22521 hybrid_clock.cc:648] HybridClock initialized: now 1786292421353686 us; error 45 us; skew 500 ppm
Aug 09 16:20:21 dist-test-slave-s11t krb5kdc[22504](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292421, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.51.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:20:21.356178 22521 init.cc:377] Logged in from keytab as kudu/127.21.51.190@KRBTEST.COM (short username kudu)
I20260809 16:20:21.356734 22521 webserver.cc:460] Webserver started at http://127.21.51.190:44551/ using document root <none> and password file <none>
I20260809 16:20:21.356984 22521 fs_manager.cc:359] Metadata directory not provided
I20260809 16:20:21.357052 22521 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:20:21.357203 22521 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:20:21.358248 22521 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786292417722232-21710-0/minicluster-data/master-0/data/instance:
uuid: "3ff8cd740c53488581a20423c907f3c2"
format_stamp: "Formatted at 2026-08-09 16:20:21 on dist-test-slave-s11t"
server_key: "171e1e7452f439e6553de9ee19658256"
server_key_iv: "eae420fd0e92e40abcb34c14f39b0ce9"
server_key_version: "encryptionkey@0"
I20260809 16:20:21.358619 22521 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786292417722232-21710-0/minicluster-data/master-0/wal/instance:
uuid: "3ff8cd740c53488581a20423c907f3c2"
format_stamp: "Formatted at 2026-08-09 16:20:21 on dist-test-slave-s11t"
server_key: "171e1e7452f439e6553de9ee19658256"
server_key_iv: "eae420fd0e92e40abcb34c14f39b0ce9"
server_key_version: "encryptionkey@0"
I20260809 16:20:21.360013 22521 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 16:20:21.360872 22548 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:20:21.361065 22521 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:20:21.361171 22521 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786292417722232-21710-0/minicluster-data/master-0/data,/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786292417722232-21710-0/minicluster-data/master-0/wal
uuid: "3ff8cd740c53488581a20423c907f3c2"
format_stamp: "Formatted at 2026-08-09 16:20:21 on dist-test-slave-s11t"
server_key: "171e1e7452f439e6553de9ee19658256"
server_key_iv: "eae420fd0e92e40abcb34c14f39b0ce9"
server_key_version: "encryptionkey@0"
I20260809 16:20:21.361255 22521 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786292417722232-21710-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786292417722232-21710-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786292417722232-21710-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:20:21.387537 22521 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:20:21.389546 22521 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:20:21.389717 22521 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:20:21.394644 22521 rpc_server.cc:307] RPC server started. Bound to: 127.21.51.190:45243
I20260809 16:20:21.394723 22636 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.51.190:45243 every 8 connection(s)
I20260809 16:20:21.395136 22521 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786292417722232-21710-0/minicluster-data/master-0/data/info.pb
I20260809 16:20:21.396270 22638 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:20:21.396989 21710 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrNqUCn/build/release/bin/kudu as pid 22521
I20260809 16:20:21.397076 21710 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786292417722232-21710-0/minicluster-data/master-0/wal/instance
I20260809 16:20:21.397251 21710 external_mini_cluster.cc:1428] Setting key 3d34345e78de13cc7f17c3c4334fa87c
I20260809 16:20:21.399821 22638 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3ff8cd740c53488581a20423c907f3c2: Bootstrap starting.
I20260809 16:20:21.400561 22638 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3ff8cd740c53488581a20423c907f3c2: Neither blocks nor log segments found. Creating new log.
I20260809 16:20:21.400936 22638 log.cc:824] T 00000000000000000000000000000000 P 3ff8cd740c53488581a20423c907f3c2: Log is configured to *not* fsync() on all Append() calls
I20260809 16:20:21.401684 22638 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3ff8cd740c53488581a20423c907f3c2: No bootstrap required, opened a new log
I20260809 16:20:21.404197 22638 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3ff8cd740c53488581a20423c907f3c2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ff8cd740c53488581a20423c907f3c2" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 45243 } }
I20260809 16:20:21.404374 22638 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3ff8cd740c53488581a20423c907f3c2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:20:21.404410 22638 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3ff8cd740c53488581a20423c907f3c2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ff8cd740c53488581a20423c907f3c2, State: Initialized, Role: FOLLOWER
I20260809 16:20:21.404560 22638 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3ff8cd740c53488581a20423c907f3c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ff8cd740c53488581a20423c907f3c2" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 45243 } }
I20260809 16:20:21.404654 22638 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3ff8cd740c53488581a20423c907f3c2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 16:20:21.404763 22638 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3ff8cd740c53488581a20423c907f3c2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 16:20:21.404817 22638 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3ff8cd740c53488581a20423c907f3c2 [term 0 FOLLOWER]: Advancing to term 1
Aug 09 16:20:21 dist-test-slave-s11t krb5kdc[22504](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292421, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.51.190@KRBTEST.COM
I20260809 16:20:21.405395 22638 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3ff8cd740c53488581a20423c907f3c2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ff8cd740c53488581a20423c907f3c2" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 45243 } }
I20260809 16:20:21.405512 22638 leader_election.cc:302] T 00000000000000000000000000000000 P 3ff8cd740c53488581a20423c907f3c2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3ff8cd740c53488581a20423c907f3c2; no voters:
I20260809 16:20:21.405704 22638 leader_election.cc:288] T 00000000000000000000000000000000 P 3ff8cd740c53488581a20423c907f3c2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 16:20:21.405745 22650 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3ff8cd740c53488581a20423c907f3c2 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:20:21.405987 22650 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3ff8cd740c53488581a20423c907f3c2 [term 1 LEADER]: Becoming Leader. State: Replica: 3ff8cd740c53488581a20423c907f3c2, State: Running, Role: LEADER
I20260809 16:20:21.405992 22638 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3ff8cd740c53488581a20423c907f3c2 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 16:20:21.406093 22650 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3ff8cd740c53488581a20423c907f3c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ff8cd740c53488581a20423c907f3c2" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 45243 } }
I20260809 16:20:21.406424 22652 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3ff8cd740c53488581a20423c907f3c2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3ff8cd740c53488581a20423c907f3c2. Latest consensus state: current_term: 1 leader_uuid: "3ff8cd740c53488581a20423c907f3c2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ff8cd740c53488581a20423c907f3c2" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 45243 } } }
I20260809 16:20:21.406615 22651 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3ff8cd740c53488581a20423c907f3c2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3ff8cd740c53488581a20423c907f3c2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ff8cd740c53488581a20423c907f3c2" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 45243 } } }
I20260809 16:20:21.406890 22651 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3ff8cd740c53488581a20423c907f3c2 [sys.catalog]: This master's current role is: LEADER
I20260809 16:20:21.406877 22652 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3ff8cd740c53488581a20423c907f3c2 [sys.catalog]: This master's current role is: LEADER
I20260809 16:20:21.407195 22664 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 16:20:21.407709 22664 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 16:20:21.409546 22645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:21.397809 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38510 (local address 127.21.51.190:45243)
0809 16:20:21.397989 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0809 16:20:21.397999 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 16:20:21.398032 (+ 33us) server_negotiation.cc:407] Connection header received
0809 16:20:21.398647 (+ 615us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:20:21.398655 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:20:21.398703 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:20:21.398866 (+ 163us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:20:21.399614 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:21.400904 (+ 1290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:21.402084 (+ 1180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:21.402723 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:21.402841 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:21.405710 (+ 2869us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:20:21.405714 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:20:21.405715 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:20:21.405727 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:20:21.407872 (+ 2145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:20:21.408346 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:20:21.408349 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:20:21.408349 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:20:21.408459 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:20:21.408672 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:20:21.408673 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:20:21.408673 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:20:21.408943 (+ 270us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:20:21.409120 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0809 16:20:21.409243 (+ 123us) server_negotiation.cc:299] Negotiation successful
0809 16:20:21.409386 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":114,"thread_start_us":82,"threads_started":1}
I20260809 16:20:21.413925 22664 catalog_manager.cc:1347] Generated new cluster ID: 7a9175fb3353464cbe1b702dd7ee2772
I20260809 16:20:21.413986 22664 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 16:20:21.440541 22664 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 16:20:21.441299 22664 catalog_manager.cc:1504] Loading token signing keys...
I20260809 16:20:21.455714 22664 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3ff8cd740c53488581a20423c907f3c2: Generated new TSK 0
I20260809 16:20:21.455953 22664 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.21.51.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.51.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.51.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.129.keytab.
Entry for principal kudu/127.21.51.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.129.keytab.
I20260809 16:20:21.491472 21710 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrNqUCn/build/release/bin/kudu
/tmp/dist-test-taskrNqUCn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786292417722232-21710-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786292417722232-21710-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786292417722232-21710-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786292417722232-21710-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.51.129:0
--local_ip_for_outbound_sockets=127.21.51.129
--webserver_interface=127.21.51.129
--webserver_port=0
--tserver_master_addrs=127.21.51.190:45243
--keytab_file=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.129.keytab
--principal=kudu/127.21.51.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.51.148:41743
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786292417722232-21710-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786292417722232-21710-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786292417722232-21710-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:20:21.574144 22678 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:20:21.574352 22678 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:20:21.574375 22678 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:20:21.575456 22678 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:20:21.575491 22678 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:20:21.575517 22678 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.51.129
I20260809 16:20:21.577054 22678 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.51.148:41743
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786292417722232-21710-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786292417722232-21710-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.129.keytab
--principal=kudu/127.21.51.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.51.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786292417722232-21710-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.51.129
--webserver_port=0
--tserver_master_addrs=127.21.51.190:45243
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22678
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.51.129
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786292417722232-21710-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:20:21.577234 22678 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:20:21.577414 22678 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:20:21.582392 22692 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:20:21.582470 22678 server_base.cc:1034] running on GCE node
W20260809 16:20:21.582394 22687 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:20:21.582392 22689 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:20:21.582873 22678 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:20:21.583069 22678 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:20:21.584208 22678 hybrid_clock.cc:648] HybridClock initialized: now 1786292421584202 us; error 31 us; skew 500 ppm
Aug 09 16:20:21 dist-test-slave-s11t krb5kdc[22504](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292421, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.51.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:20:21.586877 22678 init.cc:377] Logged in from keytab as kudu/127.21.51.129@KRBTEST.COM (short username kudu)
I20260809 16:20:21.587347 22678 webserver.cc:460] Webserver started at http://127.21.51.129:34479/ using document root <none> and password file <none>
I20260809 16:20:21.587569 22678 fs_manager.cc:359] Metadata directory not provided
I20260809 16:20:21.587613 22678 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:20:21.587780 22678 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:20:21.588897 22678 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786292417722232-21710-0/minicluster-data/ts-0/data/instance:
uuid: "589753d403ab4cf8af0bc8a34962d106"
format_stamp: "Formatted at 2026-08-09 16:20:21 on dist-test-slave-s11t"
server_key: "0bb66ea2bb754f392740465850e12320"
server_key_iv: "09a268fb5e9afd7ee326065270601d43"
server_key_version: "encryptionkey@0"
I20260809 16:20:21.589252 22678 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786292417722232-21710-0/minicluster-data/ts-0/wal/instance:
uuid: "589753d403ab4cf8af0bc8a34962d106"
format_stamp: "Formatted at 2026-08-09 16:20:21 on dist-test-slave-s11t"
server_key: "0bb66ea2bb754f392740465850e12320"
server_key_iv: "09a268fb5e9afd7ee326065270601d43"
server_key_version: "encryptionkey@0"
I20260809 16:20:21.590590 22678 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260809 16:20:21.591384 22703 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:20:21.591593 22678 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:20:21.591686 22678 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786292417722232-21710-0/minicluster-data/ts-0/data,/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786292417722232-21710-0/minicluster-data/ts-0/wal
uuid: "589753d403ab4cf8af0bc8a34962d106"
format_stamp: "Formatted at 2026-08-09 16:20:21 on dist-test-slave-s11t"
server_key: "0bb66ea2bb754f392740465850e12320"
server_key_iv: "09a268fb5e9afd7ee326065270601d43"
server_key_version: "encryptionkey@0"
I20260809 16:20:21.591759 22678 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786292417722232-21710-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786292417722232-21710-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786292417722232-21710-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:20:21.624857 22678 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:20:21.627049 22678 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:20:21.627208 22678 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:20:21.627486 22678 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:20:21.627830 22678 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:20:21.627866 22678 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:20:21.627889 22678 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:20:21.627936 22678 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:20:21.633744 22678 rpc_server.cc:307] RPC server started. Bound to: 127.21.51.129:34749
I20260809 16:20:21.633797 22888 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.51.129:34749 every 8 connection(s)
I20260809 16:20:21.634119 22678 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786292417722232-21710-0/minicluster-data/ts-0/data/info.pb
I20260809 16:20:21.636586 21710 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrNqUCn/build/release/bin/kudu as pid 22678
I20260809 16:20:21.636658 21710 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786292417722232-21710-0/minicluster-data/ts-0/wal/instance
I20260809 16:20:21.636870 21710 external_mini_cluster.cc:1428] Setting key 219c4488915f65130d6a6c727acb090a
Aug 09 16:20:21 dist-test-slave-s11t krb5kdc[22504](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292421, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.51.129@KRBTEST.COM for kudu/127.21.51.190@KRBTEST.COM
I20260809 16:20:21.645669 22645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:21.634912 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.51.129:60519 (local address 127.21.51.190:45243)
0809 16:20:21.634981 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0809 16:20:21.634983 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:20:21.635198 (+ 215us) server_negotiation.cc:407] Connection header received
0809 16:20:21.636207 (+ 1009us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:20:21.636208 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:20:21.636218 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:20:21.636276 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:20:21.637009 (+ 733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:21.637633 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:21.638668 (+ 1035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:21.638972 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:21.639041 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:21.642497 (+ 3456us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:20:21.642499 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:20:21.642499 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:20:21.642507 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:20:21.644228 (+ 1721us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:20:21.644631 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:20:21.644632 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:20:21.644633 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:20:21.644735 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:20:21.644992 (+ 257us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:20:21.644993 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:20:21.644994 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:20:21.645122 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:20:21.645201 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0809 16:20:21.645478 (+ 277us) server_negotiation.cc:299] Negotiation successful
0809 16:20:21.645567 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260809 16:20:21.645856 22891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:21.634871 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:45243 (local address 127.21.51.129:60519)
0809 16:20:21.635068 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0809 16:20:21.635110 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0809 16:20:21.636114 (+ 1004us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:20:21.636337 (+ 223us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:20:21.636340 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:20:21.636614 (+ 274us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:20:21.636915 (+ 301us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:20:21.636922 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:20:21.637737 (+ 815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:20:21.637739 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:20:21.638572 (+ 833us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:20:21.638574 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:20:21.639048 (+ 474us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:20:21.639050 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:20:21.639264 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:21.640894 (+ 1630us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:20:21.640904 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:20:21.642393 (+ 1489us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:20:21.644327 (+ 1934us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:20:21.644338 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:20:21.644339 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:20:21.644553 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:20:21.644811 (+ 258us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:20:21.644811 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:20:21.644812 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:20:21.644910 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:20:21.645206 (+ 296us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:20:21.645213 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:20:21.645345 (+ 132us) client_negotiation.cc:769] Sending connection context
0809 16:20:21.645456 (+ 111us) client_negotiation.cc:240] Negotiation successful
0809 16:20:21.645616 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":117,"threads_started":1}
I20260809 16:20:21.646387 22889 heartbeater.cc:347] Connected to a master server at 127.21.51.190:45243
I20260809 16:20:21.646502 22889 heartbeater.cc:464] Registering TS with master...
I20260809 16:20:21.646723 22889 heartbeater.cc:511] Master 127.21.51.190:45243 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.21.51.130@KRBTEST.COM; defaulting to no policy
I20260809 16:20:21.647286 22579 ts_manager.cc:194] Registered new tserver with Master: 589753d403ab4cf8af0bc8a34962d106 (127.21.51.129:34749)
I20260809 16:20:21.647920 22579 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.51.129@KRBTEST.COM'} at 127.21.51.129:60519
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.51.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.51.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.130.keytab.
Entry for principal kudu/127.21.51.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.130.keytab.
I20260809 16:20:21.665762 21710 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrNqUCn/build/release/bin/kudu
/tmp/dist-test-taskrNqUCn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786292417722232-21710-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786292417722232-21710-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786292417722232-21710-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786292417722232-21710-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.51.130:0
--local_ip_for_outbound_sockets=127.21.51.130
--webserver_interface=127.21.51.130
--webserver_port=0
--tserver_master_addrs=127.21.51.190:45243
--keytab_file=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.130.keytab
--principal=kudu/127.21.51.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.51.148:41743
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786292417722232-21710-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786292417722232-21710-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786292417722232-21710-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:20:21.757000 22896 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:20:21.757215 22896 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:20:21.757241 22896 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:20:21.758548 22896 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:20:21.758596 22896 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:20:21.758627 22896 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.51.130
I20260809 16:20:21.760288 22896 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.51.148:41743
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786292417722232-21710-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786292417722232-21710-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.130.keytab
--principal=kudu/127.21.51.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.51.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786292417722232-21710-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.51.130
--webserver_port=0
--tserver_master_addrs=127.21.51.190:45243
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22896
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.51.130
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786292417722232-21710-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:20:21.760565 22896 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:20:21.760864 22896 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:20:21.766522 22908 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:20:21.766505 22907 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:20:21.766510 22911 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:20:21.766922 22896 server_base.cc:1034] running on GCE node
I20260809 16:20:21.767163 22896 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:20:21.767482 22896 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:20:21.768656 22896 hybrid_clock.cc:648] HybridClock initialized: now 1786292421768651 us; error 38 us; skew 500 ppm
Aug 09 16:20:21 dist-test-slave-s11t krb5kdc[22504](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292421, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.51.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:20:21.771369 22896 init.cc:377] Logged in from keytab as kudu/127.21.51.130@KRBTEST.COM (short username kudu)
I20260809 16:20:21.771921 22896 webserver.cc:460] Webserver started at http://127.21.51.130:45585/ using document root <none> and password file <none>
I20260809 16:20:21.772174 22896 fs_manager.cc:359] Metadata directory not provided
I20260809 16:20:21.772250 22896 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:20:21.772416 22896 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:20:21.773623 22896 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786292417722232-21710-0/minicluster-data/ts-1/data/instance:
uuid: "2e3534dcf6a842b5948550fd16a1b335"
format_stamp: "Formatted at 2026-08-09 16:20:21 on dist-test-slave-s11t"
server_key: "a544d9443e3d7920ce629b834e2b2f98"
server_key_iv: "3198253e0433b201e7fa36111cdf472d"
server_key_version: "encryptionkey@0"
I20260809 16:20:21.774046 22896 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786292417722232-21710-0/minicluster-data/ts-1/wal/instance:
uuid: "2e3534dcf6a842b5948550fd16a1b335"
format_stamp: "Formatted at 2026-08-09 16:20:21 on dist-test-slave-s11t"
server_key: "a544d9443e3d7920ce629b834e2b2f98"
server_key_iv: "3198253e0433b201e7fa36111cdf472d"
server_key_version: "encryptionkey@0"
I20260809 16:20:21.775466 22896 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260809 16:20:21.776257 22921 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:20:21.776440 22896 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 16:20:21.776551 22896 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786292417722232-21710-0/minicluster-data/ts-1/data,/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786292417722232-21710-0/minicluster-data/ts-1/wal
uuid: "2e3534dcf6a842b5948550fd16a1b335"
format_stamp: "Formatted at 2026-08-09 16:20:21 on dist-test-slave-s11t"
server_key: "a544d9443e3d7920ce629b834e2b2f98"
server_key_iv: "3198253e0433b201e7fa36111cdf472d"
server_key_version: "encryptionkey@0"
I20260809 16:20:21.776641 22896 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786292417722232-21710-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786292417722232-21710-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786292417722232-21710-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:20:21.790373 22896 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:20:21.792245 22896 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:20:21.792418 22896 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:20:21.792657 22896 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:20:21.793020 22896 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:20:21.793082 22896 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:20:21.793128 22896 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:20:21.793177 22896 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:20:21.800100 22896 rpc_server.cc:307] RPC server started. Bound to: 127.21.51.130:37709
I20260809 16:20:21.800135 23096 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.51.130:37709 every 8 connection(s)
I20260809 16:20:21.800489 22896 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786292417722232-21710-0/minicluster-data/ts-1/data/info.pb
I20260809 16:20:21.801694 21710 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrNqUCn/build/release/bin/kudu as pid 22896
I20260809 16:20:21.801767 21710 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786292417722232-21710-0/minicluster-data/ts-1/wal/instance
I20260809 16:20:21.801939 21710 external_mini_cluster.cc:1428] Setting key 8f6ef36e1417530ae448b1a9640105b2
Aug 09 16:20:21 dist-test-slave-s11t krb5kdc[22504](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292421, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.51.130@KRBTEST.COM for kudu/127.21.51.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.21.51.131@KRBTEST.COM; defaulting to no policy
I20260809 16:20:21.812489 22645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:21.801285 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.51.130:47731 (local address 127.21.51.190:45243)
0809 16:20:21.801328 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0809 16:20:21.801331 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 16:20:21.801574 (+ 243us) server_negotiation.cc:407] Connection header received
0809 16:20:21.802816 (+ 1242us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:20:21.802817 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:20:21.802829 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:20:21.802905 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:20:21.803509 (+ 604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:21.804266 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:21.806160 (+ 1894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:21.806510 (+ 350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:21.806579 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:21.809057 (+ 2478us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:20:21.809059 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:20:21.809059 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:20:21.809067 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:20:21.810905 (+ 1838us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:20:21.811384 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:20:21.811386 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:20:21.811386 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:20:21.811453 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:20:21.811719 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:20:21.811720 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:20:21.811721 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:20:21.811870 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:20:21.811950 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0809 16:20:21.812292 (+ 342us) server_negotiation.cc:299] Negotiation successful
0809 16:20:21.812386 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260809 16:20:21.812721 23099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:21.801257 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:45243 (local address 127.21.51.130:47731)
0809 16:20:21.801430 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0809 16:20:21.801474 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 16:20:21.802706 (+ 1232us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:20:21.802960 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:20:21.802963 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:20:21.803224 (+ 261us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:20:21.803421 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:20:21.803425 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:20:21.804375 (+ 950us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:20:21.804377 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:20:21.806047 (+ 1670us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:20:21.806050 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:20:21.806612 (+ 562us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:20:21.806613 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:20:21.806722 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:21.807318 (+ 596us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:20:21.807329 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:20:21.808957 (+ 1628us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:20:21.811008 (+ 2051us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:20:21.811015 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:20:21.811017 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:20:21.811292 (+ 275us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:20:21.811535 (+ 243us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:20:21.811536 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:20:21.811536 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:20:21.811645 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:20:21.811951 (+ 306us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:20:21.811956 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:20:21.812078 (+ 122us) client_negotiation.cc:769] Sending connection context
0809 16:20:21.812261 (+ 183us) client_negotiation.cc:240] Negotiation successful
0809 16:20:21.812437 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":105,"threads_started":1}
I20260809 16:20:21.813251 23097 heartbeater.cc:347] Connected to a master server at 127.21.51.190:45243
I20260809 16:20:21.813349 23097 heartbeater.cc:464] Registering TS with master...
I20260809 16:20:21.813499 23097 heartbeater.cc:511] Master 127.21.51.190:45243 requested a full tablet report, sending...
I20260809 16:20:21.813892 22579 ts_manager.cc:194] Registered new tserver with Master: 2e3534dcf6a842b5948550fd16a1b335 (127.21.51.130:37709)
I20260809 16:20:21.814316 22579 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.51.130@KRBTEST.COM'} at 127.21.51.130:47731
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.51.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.51.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.131.keytab.
Entry for principal kudu/127.21.51.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.131.keytab.
I20260809 16:20:21.831552 21710 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrNqUCn/build/release/bin/kudu
/tmp/dist-test-taskrNqUCn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786292417722232-21710-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786292417722232-21710-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786292417722232-21710-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786292417722232-21710-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.51.131:0
--local_ip_for_outbound_sockets=127.21.51.131
--webserver_interface=127.21.51.131
--webserver_port=0
--tserver_master_addrs=127.21.51.190:45243
--keytab_file=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.131.keytab
--principal=kudu/127.21.51.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.51.148:41743
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786292417722232-21710-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786292417722232-21710-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786292417722232-21710-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:20:21.915813 23103 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:20:21.916011 23103 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:20:21.916034 23103 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:20:21.917266 23103 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:20:21.917310 23103 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:20:21.917338 23103 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.51.131
I20260809 16:20:21.918748 23103 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.51.148:41743
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786292417722232-21710-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786292417722232-21710-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.131.keytab
--principal=kudu/127.21.51.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.51.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786292417722232-21710-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.51.131
--webserver_port=0
--tserver_master_addrs=127.21.51.190:45243
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23103
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.51.131
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786292417722232-21710-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:20:21.918923 23103 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:20:21.919091 23103 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:20:21.924103 23112 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:20:21.924212 23103 server_base.cc:1034] running on GCE node
W20260809 16:20:21.924093 23110 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:20:21.924091 23117 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:20:21.924515 23103 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:20:21.924769 23103 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:20:21.925949 23103 hybrid_clock.cc:648] HybridClock initialized: now 1786292421925939 us; error 26 us; skew 500 ppm
Aug 09 16:20:21 dist-test-slave-s11t krb5kdc[22504](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292421, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.51.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:20:21.928424 23103 init.cc:377] Logged in from keytab as kudu/127.21.51.131@KRBTEST.COM (short username kudu)
I20260809 16:20:21.928965 23103 webserver.cc:460] Webserver started at http://127.21.51.131:43129/ using document root <none> and password file <none>
I20260809 16:20:21.929194 23103 fs_manager.cc:359] Metadata directory not provided
I20260809 16:20:21.929266 23103 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:20:21.929419 23103 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:20:21.930436 23103 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786292417722232-21710-0/minicluster-data/ts-2/data/instance:
uuid: "30ad388f114948c38017799df2e20dc5"
format_stamp: "Formatted at 2026-08-09 16:20:21 on dist-test-slave-s11t"
server_key: "f9991f76a12fccefee27e0d6160061a7"
server_key_iv: "3cb794c3762edf02f5cc95740dac0d06"
server_key_version: "encryptionkey@0"
I20260809 16:20:21.930810 23103 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786292417722232-21710-0/minicluster-data/ts-2/wal/instance:
uuid: "30ad388f114948c38017799df2e20dc5"
format_stamp: "Formatted at 2026-08-09 16:20:21 on dist-test-slave-s11t"
server_key: "f9991f76a12fccefee27e0d6160061a7"
server_key_iv: "3cb794c3762edf02f5cc95740dac0d06"
server_key_version: "encryptionkey@0"
I20260809 16:20:21.932125 23103 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260809 16:20:21.932966 23127 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:20:21.933178 23103 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:20:21.933288 23103 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786292417722232-21710-0/minicluster-data/ts-2/data,/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786292417722232-21710-0/minicluster-data/ts-2/wal
uuid: "30ad388f114948c38017799df2e20dc5"
format_stamp: "Formatted at 2026-08-09 16:20:21 on dist-test-slave-s11t"
server_key: "f9991f76a12fccefee27e0d6160061a7"
server_key_iv: "3cb794c3762edf02f5cc95740dac0d06"
server_key_version: "encryptionkey@0"
I20260809 16:20:21.933368 23103 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786292417722232-21710-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786292417722232-21710-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786292417722232-21710-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:20:21.946975 23103 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:20:21.949137 23103 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:20:21.949309 23103 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:20:21.949558 23103 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:20:21.949898 23103 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:20:21.949955 23103 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:20:21.949992 23103 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:20:21.950040 23103 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:20:21.956007 23103 rpc_server.cc:307] RPC server started. Bound to: 127.21.51.131:35859
I20260809 16:20:21.956053 23295 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.51.131:35859 every 8 connection(s)
I20260809 16:20:21.956444 23103 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786292417722232-21710-0/minicluster-data/ts-2/data/info.pb
I20260809 16:20:21.956945 21710 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrNqUCn/build/release/bin/kudu as pid 23103
I20260809 16:20:21.957073 21710 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786292417722232-21710-0/minicluster-data/ts-2/wal/instance
I20260809 16:20:21.957273 21710 external_mini_cluster.cc:1428] Setting key d3b3355c8b05e6c5c40dcafc3c2a4b8d
Aug 09 16:20:21 dist-test-slave-s11t krb5kdc[22504](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292421, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.51.131@KRBTEST.COM for kudu/127.21.51.190@KRBTEST.COM
I20260809 16:20:21.966220 22645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:21.957204 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.51.131:49003 (local address 127.21.51.190:45243)
0809 16:20:21.957261 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0809 16:20:21.957263 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:20:21.957508 (+ 245us) server_negotiation.cc:407] Connection header received
0809 16:20:21.958594 (+ 1086us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:20:21.958595 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:20:21.958607 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:20:21.958651 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:20:21.959288 (+ 637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:21.959811 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:21.960895 (+ 1084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:21.961112 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:21.961191 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:21.963491 (+ 2300us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:20:21.963493 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:20:21.963493 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:20:21.963500 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:20:21.965073 (+ 1573us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:20:21.965442 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:20:21.965443 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:20:21.965443 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:20:21.965493 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:20:21.965705 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:20:21.965706 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:20:21.965706 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:20:21.965805 (+ 99us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:20:21.965855 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0809 16:20:21.966093 (+ 238us) server_negotiation.cc:299] Negotiation successful
0809 16:20:21.966153 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260809 16:20:21.966455 23298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:21.957205 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:45243 (local address 127.21.51.131:49003)
0809 16:20:21.957390 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0809 16:20:21.957432 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0809 16:20:21.958487 (+ 1055us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:20:21.958727 (+ 240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:20:21.958733 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:20:21.958986 (+ 253us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:20:21.959170 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:20:21.959176 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:20:21.959912 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:20:21.959914 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:20:21.960809 (+ 895us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:20:21.960811 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:20:21.961196 (+ 385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:20:21.961197 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:20:21.961332 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:21.961830 (+ 498us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:20:21.961838 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:20:21.963402 (+ 1564us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:20:21.965145 (+ 1743us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:20:21.965151 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:20:21.965153 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:20:21.965370 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:20:21.965552 (+ 182us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:20:21.965552 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:20:21.965553 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:20:21.965649 (+ 96us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:20:21.965860 (+ 211us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:20:21.965864 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:20:21.965982 (+ 118us) client_negotiation.cc:769] Sending connection context
0809 16:20:21.966066 (+ 84us) client_negotiation.cc:240] Negotiation successful
0809 16:20:21.966195 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":114,"threads_started":1}
I20260809 16:20:21.966934 23296 heartbeater.cc:347] Connected to a master server at 127.21.51.190:45243
I20260809 16:20:21.967017 23296 heartbeater.cc:464] Registering TS with master...
I20260809 16:20:21.967211 23296 heartbeater.cc:511] Master 127.21.51.190:45243 requested a full tablet report, sending...
I20260809 16:20:21.967610 22579 ts_manager.cc:194] Registered new tserver with Master: 30ad388f114948c38017799df2e20dc5 (127.21.51.131:35859)
I20260809 16:20:21.967962 22579 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.51.131@KRBTEST.COM'} at 127.21.51.131:49003
I20260809 16:20:21.969087 21710 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 16:20:21.976758 22645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:21.970594 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38526 (local address 127.21.51.190:45243)
0809 16:20:21.970631 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0809 16:20:21.970633 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:20:21.970759 (+ 126us) server_negotiation.cc:407] Connection header received
0809 16:20:21.970850 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:20:21.970851 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:20:21.970861 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:20:21.970911 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:20:21.971445 (+ 534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:21.971916 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:21.972632 (+ 716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:21.972874 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:21.972963 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:21.974124 (+ 1161us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:20:21.974126 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:20:21.974126 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:20:21.974134 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:20:21.975549 (+ 1415us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:20:21.975917 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:20:21.975918 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:20:21.975918 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:20:21.975982 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:20:21.976186 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:20:21.976187 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:20:21.976188 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:20:21.976335 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:20:21.976447 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0809 16:20:21.976553 (+ 106us) server_negotiation.cc:299] Negotiation successful
0809 16:20:21.976635 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260809 16:20:21.978941 22579 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38526:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 16:20:21.980623 22579 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 16:20:21.987450 23323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:21.983238 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.130:37709 (local address 127.0.0.1:57232)
0809 16:20:21.983365 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0809 16:20:21.983374 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 16:20:21.983429 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:20:21.983735 (+ 306us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:20:21.983739 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:20:21.983741 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:20:21.983808 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:20:21.983810 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:20:21.984970 (+ 1160us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:20:21.984971 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:20:21.986198 (+ 1227us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:20:21.986201 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:20:21.987113 (+ 912us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:20:21.987114 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:20:21.987214 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:21.987231 (+ 17us) client_negotiation.cc:769] Sending connection context
0809 16:20:21.987306 (+ 75us) client_negotiation.cc:240] Negotiation successful
0809 16:20:21.987368 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":94,"threads_started":1}
I20260809 16:20:21.987617 23325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:21.983433 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37534 (local address 127.21.51.129:34749)
0809 16:20:21.983607 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0809 16:20:21.983611 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 16:20:21.983622 (+ 11us) server_negotiation.cc:407] Connection header received
0809 16:20:21.983662 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:20:21.983665 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:20:21.983681 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:20:21.983940 (+ 259us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:20:21.984029 (+ 89us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:21.985185 (+ 1156us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:21.986483 (+ 1298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:21.987141 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:21.987240 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:21.987291 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0809 16:20:21.987499 (+ 208us) server_negotiation.cc:299] Negotiation successful
0809 16:20:21.987539 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":123,"thread_start_us":85,"threads_started":1}
I20260809 16:20:21.987751 23319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:21.983008 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.129:34749 (local address 127.0.0.1:37534)
0809 16:20:21.983140 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0809 16:20:21.983180 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0809 16:20:21.983248 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:20:21.983791 (+ 543us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:20:21.983792 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:20:21.983795 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:20:21.983856 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:20:21.983858 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:20:21.985313 (+ 1455us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:20:21.985314 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:20:21.986408 (+ 1094us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:20:21.986410 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:20:21.987266 (+ 856us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:20:21.987266 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:20:21.987327 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:21.987330 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 16:20:21.987402 (+ 72us) client_negotiation.cc:240] Negotiation successful
0809 16:20:21.987498 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":103,"mutex_wait_us":32,"thread_start_us":79,"threads_started":1}
I20260809 16:20:21.987939 23324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:21.983367 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57232 (local address 127.21.51.130:37709)
0809 16:20:21.983545 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0809 16:20:21.983549 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 16:20:21.983560 (+ 11us) server_negotiation.cc:407] Connection header received
0809 16:20:21.983603 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:20:21.983605 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:20:21.983618 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:20:21.983703 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:20:21.983874 (+ 171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:21.984865 (+ 991us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:21.986294 (+ 1429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:21.987028 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:21.987102 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:21.987160 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0809 16:20:21.987781 (+ 621us) server_negotiation.cc:299] Negotiation successful
0809 16:20:21.987834 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":145,"thread_start_us":111,"threads_started":1}
I20260809 16:20:21.988348 23322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:21.983043 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.131:35859 (local address 127.0.0.1:33312)
0809 16:20:21.983184 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0809 16:20:21.983196 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 16:20:21.983250 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:20:21.984052 (+ 802us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:20:21.984053 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:20:21.984056 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:20:21.984103 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:20:21.984105 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:20:21.985337 (+ 1232us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:20:21.985338 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:20:21.986210 (+ 872us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:20:21.986212 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:20:21.988117 (+ 1905us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:20:21.988119 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:20:21.988185 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:21.988190 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 16:20:21.988234 (+ 44us) client_negotiation.cc:240] Negotiation successful
0809 16:20:21.988274 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"mutex_wait_us":1,"thread_start_us":96,"threads_started":1}
I20260809 16:20:21.988395 23326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:21.983624 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33312 (local address 127.21.51.131:35859)
0809 16:20:21.983800 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0809 16:20:21.983804 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 16:20:21.983812 (+ 8us) server_negotiation.cc:407] Connection header received
0809 16:20:21.983925 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:20:21.983928 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:20:21.983944 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:20:21.984049 (+ 105us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:20:21.984182 (+ 133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:21.985221 (+ 1039us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:21.987197 (+ 1976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:21.987977 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:21.988192 (+ 215us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:21.988255 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0809 16:20:21.988279 (+ 24us) server_negotiation.cc:299] Negotiation successful
0809 16:20:21.988317 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":150,"thread_start_us":81,"threads_started":1}
I20260809 16:20:21.988492 22772 tablet_service.cc:1467] Processing CreateTablet for tablet 0f4371da054442339e8e57da36f544d8 (DEFAULT_TABLE table=test-table [id=b811efb4a20444a78b3c157c077eeed7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:20:21.988935 22772 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0f4371da054442339e8e57da36f544d8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:20:21.989076 23194 tablet_service.cc:1467] Processing CreateTablet for tablet 0f4371da054442339e8e57da36f544d8 (DEFAULT_TABLE table=test-table [id=b811efb4a20444a78b3c157c077eeed7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:20:21.989390 23194 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0f4371da054442339e8e57da36f544d8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:20:21.992395 23327 tablet_bootstrap.cc:492] T 0f4371da054442339e8e57da36f544d8 P 589753d403ab4cf8af0bc8a34962d106: Bootstrap starting.
I20260809 16:20:21.992597 23329 tablet_bootstrap.cc:492] T 0f4371da054442339e8e57da36f544d8 P 30ad388f114948c38017799df2e20dc5: Bootstrap starting.
I20260809 16:20:21.992884 22995 tablet_service.cc:1467] Processing CreateTablet for tablet 0f4371da054442339e8e57da36f544d8 (DEFAULT_TABLE table=test-table [id=b811efb4a20444a78b3c157c077eeed7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:20:21.993186 22995 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0f4371da054442339e8e57da36f544d8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:20:21.993186 23327 tablet_bootstrap.cc:654] T 0f4371da054442339e8e57da36f544d8 P 589753d403ab4cf8af0bc8a34962d106: Neither blocks nor log segments found. Creating new log.
I20260809 16:20:21.993242 23329 tablet_bootstrap.cc:654] T 0f4371da054442339e8e57da36f544d8 P 30ad388f114948c38017799df2e20dc5: Neither blocks nor log segments found. Creating new log.
I20260809 16:20:21.993856 23327 log.cc:824] T 0f4371da054442339e8e57da36f544d8 P 589753d403ab4cf8af0bc8a34962d106: Log is configured to *not* fsync() on all Append() calls
I20260809 16:20:21.993897 23329 log.cc:824] T 0f4371da054442339e8e57da36f544d8 P 30ad388f114948c38017799df2e20dc5: Log is configured to *not* fsync() on all Append() calls
I20260809 16:20:21.994689 23329 tablet_bootstrap.cc:492] T 0f4371da054442339e8e57da36f544d8 P 30ad388f114948c38017799df2e20dc5: No bootstrap required, opened a new log
I20260809 16:20:21.994695 23327 tablet_bootstrap.cc:492] T 0f4371da054442339e8e57da36f544d8 P 589753d403ab4cf8af0bc8a34962d106: No bootstrap required, opened a new log
I20260809 16:20:21.994760 23329 ts_tablet_manager.cc:1397] T 0f4371da054442339e8e57da36f544d8 P 30ad388f114948c38017799df2e20dc5: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 16:20:21.994808 23327 ts_tablet_manager.cc:1397] T 0f4371da054442339e8e57da36f544d8 P 589753d403ab4cf8af0bc8a34962d106: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260809 16:20:21.996285 23329 raft_consensus.cc:348] T 0f4371da054442339e8e57da36f544d8 P 30ad388f114948c38017799df2e20dc5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "589753d403ab4cf8af0bc8a34962d106" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 34749 } } peers { permanent_uuid: "30ad388f114948c38017799df2e20dc5" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 35859 } } peers { permanent_uuid: "2e3534dcf6a842b5948550fd16a1b335" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 37709 } }
I20260809 16:20:21.996462 23332 tablet_bootstrap.cc:492] T 0f4371da054442339e8e57da36f544d8 P 2e3534dcf6a842b5948550fd16a1b335: Bootstrap starting.
I20260809 16:20:21.996496 23329 raft_consensus.cc:374] T 0f4371da054442339e8e57da36f544d8 P 30ad388f114948c38017799df2e20dc5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:20:21.996541 23329 raft_consensus.cc:729] T 0f4371da054442339e8e57da36f544d8 P 30ad388f114948c38017799df2e20dc5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 30ad388f114948c38017799df2e20dc5, State: Initialized, Role: FOLLOWER
I20260809 16:20:21.996673 23329 consensus_queue.cc:260] T 0f4371da054442339e8e57da36f544d8 P 30ad388f114948c38017799df2e20dc5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "589753d403ab4cf8af0bc8a34962d106" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 34749 } } peers { permanent_uuid: "30ad388f114948c38017799df2e20dc5" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 35859 } } peers { permanent_uuid: "2e3534dcf6a842b5948550fd16a1b335" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 37709 } }
I20260809 16:20:21.997007 23329 ts_tablet_manager.cc:1428] T 0f4371da054442339e8e57da36f544d8 P 30ad388f114948c38017799df2e20dc5: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260809 16:20:21.997115 23332 tablet_bootstrap.cc:654] T 0f4371da054442339e8e57da36f544d8 P 2e3534dcf6a842b5948550fd16a1b335: Neither blocks nor log segments found. Creating new log.
I20260809 16:20:21.997136 23296 heartbeater.cc:503] Master 127.21.51.190:45243 was elected leader, sending a full tablet report...
I20260809 16:20:21.997246 23327 raft_consensus.cc:348] T 0f4371da054442339e8e57da36f544d8 P 589753d403ab4cf8af0bc8a34962d106 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "589753d403ab4cf8af0bc8a34962d106" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 34749 } } peers { permanent_uuid: "30ad388f114948c38017799df2e20dc5" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 35859 } } peers { permanent_uuid: "2e3534dcf6a842b5948550fd16a1b335" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 37709 } }
I20260809 16:20:21.997455 23327 raft_consensus.cc:374] T 0f4371da054442339e8e57da36f544d8 P 589753d403ab4cf8af0bc8a34962d106 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:20:21.997521 23327 raft_consensus.cc:729] T 0f4371da054442339e8e57da36f544d8 P 589753d403ab4cf8af0bc8a34962d106 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 589753d403ab4cf8af0bc8a34962d106, State: Initialized, Role: FOLLOWER
I20260809 16:20:21.997668 23327 consensus_queue.cc:260] T 0f4371da054442339e8e57da36f544d8 P 589753d403ab4cf8af0bc8a34962d106 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "589753d403ab4cf8af0bc8a34962d106" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 34749 } } peers { permanent_uuid: "30ad388f114948c38017799df2e20dc5" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 35859 } } peers { permanent_uuid: "2e3534dcf6a842b5948550fd16a1b335" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 37709 } }
I20260809 16:20:21.997718 23332 log.cc:824] T 0f4371da054442339e8e57da36f544d8 P 2e3534dcf6a842b5948550fd16a1b335: Log is configured to *not* fsync() on all Append() calls
I20260809 16:20:21.997993 23327 ts_tablet_manager.cc:1428] T 0f4371da054442339e8e57da36f544d8 P 589753d403ab4cf8af0bc8a34962d106: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260809 16:20:21.998189 22889 heartbeater.cc:503] Master 127.21.51.190:45243 was elected leader, sending a full tablet report...
I20260809 16:20:21.998699 23332 tablet_bootstrap.cc:492] T 0f4371da054442339e8e57da36f544d8 P 2e3534dcf6a842b5948550fd16a1b335: No bootstrap required, opened a new log
I20260809 16:20:21.998769 23332 ts_tablet_manager.cc:1397] T 0f4371da054442339e8e57da36f544d8 P 2e3534dcf6a842b5948550fd16a1b335: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260809 16:20:22.000124 23332 raft_consensus.cc:348] T 0f4371da054442339e8e57da36f544d8 P 2e3534dcf6a842b5948550fd16a1b335 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "589753d403ab4cf8af0bc8a34962d106" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 34749 } } peers { permanent_uuid: "30ad388f114948c38017799df2e20dc5" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 35859 } } peers { permanent_uuid: "2e3534dcf6a842b5948550fd16a1b335" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 37709 } }
I20260809 16:20:22.000247 23332 raft_consensus.cc:374] T 0f4371da054442339e8e57da36f544d8 P 2e3534dcf6a842b5948550fd16a1b335 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:20:22.000268 23332 raft_consensus.cc:729] T 0f4371da054442339e8e57da36f544d8 P 2e3534dcf6a842b5948550fd16a1b335 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2e3534dcf6a842b5948550fd16a1b335, State: Initialized, Role: FOLLOWER
I20260809 16:20:22.000371 23332 consensus_queue.cc:260] T 0f4371da054442339e8e57da36f544d8 P 2e3534dcf6a842b5948550fd16a1b335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "589753d403ab4cf8af0bc8a34962d106" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 34749 } } peers { permanent_uuid: "30ad388f114948c38017799df2e20dc5" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 35859 } } peers { permanent_uuid: "2e3534dcf6a842b5948550fd16a1b335" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 37709 } }
I20260809 16:20:22.000605 23332 ts_tablet_manager.cc:1428] T 0f4371da054442339e8e57da36f544d8 P 2e3534dcf6a842b5948550fd16a1b335: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260809 16:20:22.000658 23097 heartbeater.cc:503] Master 127.21.51.190:45243 was elected leader, sending a full tablet report...
W20260809 16:20:22.051190 23098 tablet.cc:2369] T 0f4371da054442339e8e57da36f544d8 P 2e3534dcf6a842b5948550fd16a1b335: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 16:20:22.134835 22890 tablet.cc:2369] T 0f4371da054442339e8e57da36f544d8 P 589753d403ab4cf8af0bc8a34962d106: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 16:20:22.171281 23335 raft_consensus.cc:482] T 0f4371da054442339e8e57da36f544d8 P 589753d403ab4cf8af0bc8a34962d106 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 16:20:22.171406 23335 raft_consensus.cc:504] T 0f4371da054442339e8e57da36f544d8 P 589753d403ab4cf8af0bc8a34962d106 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "589753d403ab4cf8af0bc8a34962d106" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 34749 } } peers { permanent_uuid: "30ad388f114948c38017799df2e20dc5" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 35859 } } peers { permanent_uuid: "2e3534dcf6a842b5948550fd16a1b335" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 37709 } }
I20260809 16:20:22.171733 23335 leader_election.cc:288] T 0f4371da054442339e8e57da36f544d8 P 589753d403ab4cf8af0bc8a34962d106 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 30ad388f114948c38017799df2e20dc5 (127.21.51.131:35859), 2e3534dcf6a842b5948550fd16a1b335 (127.21.51.130:37709)
I20260809 16:20:22.174878 23342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:22.171916 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.130:37709 (local address 127.21.51.129:44129)
0809 16:20:22.172065 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0809 16:20:22.172078 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 16:20:22.172142 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:20:22.172294 (+ 152us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:20:22.172295 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:20:22.172297 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:20:22.172425 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:20:22.172427 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:20:22.173136 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:20:22.173138 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:20:22.174001 (+ 863us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:20:22.174004 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:20:22.174581 (+ 577us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:20:22.174582 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:20:22.174667 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:22.174670 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 16:20:22.174736 (+ 66us) client_negotiation.cc:240] Negotiation successful
0809 16:20:22.174778 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"mutex_wait_us":52,"thread_start_us":69,"threads_started":1}
I20260809 16:20:22.174865 23324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:22.172007 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.51.129:44129 (local address 127.21.51.130:37709)
0809 16:20:22.172096 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0809 16:20:22.172099 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 16:20:22.172130 (+ 31us) server_negotiation.cc:407] Connection header received
0809 16:20:22.172221 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:20:22.172222 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:20:22.172231 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:20:22.172288 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:20:22.172491 (+ 203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:22.173054 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:22.174100 (+ 1046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:22.174477 (+ 377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:22.174573 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:22.174584 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0809 16:20:22.174744 (+ 160us) server_negotiation.cc:299] Negotiation successful
0809 16:20:22.174781 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260809 16:20:22.174865 23326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:22.172026 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.51.129:53135 (local address 127.21.51.131:35859)
0809 16:20:22.172078 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0809 16:20:22.172080 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:20:22.172195 (+ 115us) server_negotiation.cc:407] Connection header received
0809 16:20:22.172264 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:20:22.172265 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:20:22.172275 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:20:22.172350 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:20:22.172535 (+ 185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:22.173068 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:22.174092 (+ 1024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:22.174494 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:22.174580 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:22.174590 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0809 16:20:22.174755 (+ 165us) server_negotiation.cc:299] Negotiation successful
0809 16:20:22.174794 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260809 16:20:22.174893 23340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:22.171918 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.131:35859 (local address 127.21.51.129:53135)
0809 16:20:22.172099 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0809 16:20:22.172110 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 16:20:22.172206 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:20:22.172352 (+ 146us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:20:22.172353 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:20:22.172355 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:20:22.172451 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:20:22.172453 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:20:22.173150 (+ 697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:20:22.173151 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:20:22.173994 (+ 843us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:20:22.173997 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:20:22.174593 (+ 596us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:20:22.174595 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:20:22.174669 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:22.174676 (+ 7us) client_negotiation.cc:769] Sending connection context
0809 16:20:22.174750 (+ 74us) client_negotiation.cc:240] Negotiation successful
0809 16:20:22.174793 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":165,"mutex_wait_us":2,"thread_start_us":63,"threads_started":1}
I20260809 16:20:22.175309 23023 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f4371da054442339e8e57da36f544d8" candidate_uuid: "589753d403ab4cf8af0bc8a34962d106" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2e3534dcf6a842b5948550fd16a1b335" is_pre_election: true
I20260809 16:20:22.175508 23023 raft_consensus.cc:2454] T 0f4371da054442339e8e57da36f544d8 P 2e3534dcf6a842b5948550fd16a1b335 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 589753d403ab4cf8af0bc8a34962d106 in term 0.
I20260809 16:20:22.175443 23227 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f4371da054442339e8e57da36f544d8" candidate_uuid: "589753d403ab4cf8af0bc8a34962d106" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30ad388f114948c38017799df2e20dc5" is_pre_election: true
I20260809 16:20:22.175650 23227 raft_consensus.cc:2454] T 0f4371da054442339e8e57da36f544d8 P 30ad388f114948c38017799df2e20dc5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 589753d403ab4cf8af0bc8a34962d106 in term 0.
I20260809 16:20:22.175773 22705 leader_election.cc:302] T 0f4371da054442339e8e57da36f544d8 P 589753d403ab4cf8af0bc8a34962d106 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2e3534dcf6a842b5948550fd16a1b335, 589753d403ab4cf8af0bc8a34962d106; no voters:
I20260809 16:20:22.175926 23335 raft_consensus.cc:2781] T 0f4371da054442339e8e57da36f544d8 P 589753d403ab4cf8af0bc8a34962d106 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 16:20:22.176007 23335 raft_consensus.cc:482] T 0f4371da054442339e8e57da36f544d8 P 589753d403ab4cf8af0bc8a34962d106 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 16:20:22.176043 23335 raft_consensus.cc:3037] T 0f4371da054442339e8e57da36f544d8 P 589753d403ab4cf8af0bc8a34962d106 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:20:22.176854 23335 raft_consensus.cc:504] T 0f4371da054442339e8e57da36f544d8 P 589753d403ab4cf8af0bc8a34962d106 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "589753d403ab4cf8af0bc8a34962d106" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 34749 } } peers { permanent_uuid: "30ad388f114948c38017799df2e20dc5" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 35859 } } peers { permanent_uuid: "2e3534dcf6a842b5948550fd16a1b335" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 37709 } }
I20260809 16:20:22.177001 23335 leader_election.cc:288] T 0f4371da054442339e8e57da36f544d8 P 589753d403ab4cf8af0bc8a34962d106 [CANDIDATE]: Term 1 election: Requested vote from peers 30ad388f114948c38017799df2e20dc5 (127.21.51.131:35859), 2e3534dcf6a842b5948550fd16a1b335 (127.21.51.130:37709)
I20260809 16:20:22.177199 23227 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f4371da054442339e8e57da36f544d8" candidate_uuid: "589753d403ab4cf8af0bc8a34962d106" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30ad388f114948c38017799df2e20dc5"
I20260809 16:20:22.177271 23227 raft_consensus.cc:3037] T 0f4371da054442339e8e57da36f544d8 P 30ad388f114948c38017799df2e20dc5 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:20:22.178057 23227 raft_consensus.cc:2454] T 0f4371da054442339e8e57da36f544d8 P 30ad388f114948c38017799df2e20dc5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 589753d403ab4cf8af0bc8a34962d106 in term 1.
I20260809 16:20:22.178182 23023 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f4371da054442339e8e57da36f544d8" candidate_uuid: "589753d403ab4cf8af0bc8a34962d106" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2e3534dcf6a842b5948550fd16a1b335"
I20260809 16:20:22.178238 22704 leader_election.cc:302] T 0f4371da054442339e8e57da36f544d8 P 589753d403ab4cf8af0bc8a34962d106 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 30ad388f114948c38017799df2e20dc5, 589753d403ab4cf8af0bc8a34962d106; no voters:
I20260809 16:20:22.178272 23023 raft_consensus.cc:3037] T 0f4371da054442339e8e57da36f544d8 P 2e3534dcf6a842b5948550fd16a1b335 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:20:22.178319 23335 raft_consensus.cc:2781] T 0f4371da054442339e8e57da36f544d8 P 589753d403ab4cf8af0bc8a34962d106 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:20:22.178550 23335 raft_consensus.cc:686] T 0f4371da054442339e8e57da36f544d8 P 589753d403ab4cf8af0bc8a34962d106 [term 1 LEADER]: Becoming Leader. State: Replica: 589753d403ab4cf8af0bc8a34962d106, State: Running, Role: LEADER
I20260809 16:20:22.178651 23335 consensus_queue.cc:237] T 0f4371da054442339e8e57da36f544d8 P 589753d403ab4cf8af0bc8a34962d106 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "589753d403ab4cf8af0bc8a34962d106" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 34749 } } peers { permanent_uuid: "30ad388f114948c38017799df2e20dc5" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 35859 } } peers { permanent_uuid: "2e3534dcf6a842b5948550fd16a1b335" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 37709 } }
I20260809 16:20:22.179256 23023 raft_consensus.cc:2454] T 0f4371da054442339e8e57da36f544d8 P 2e3534dcf6a842b5948550fd16a1b335 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 589753d403ab4cf8af0bc8a34962d106 in term 1.
I20260809 16:20:22.179610 22578 catalog_manager.cc:5495] T 0f4371da054442339e8e57da36f544d8 P 589753d403ab4cf8af0bc8a34962d106 reported cstate change: term changed from 0 to 1, leader changed from <none> to 589753d403ab4cf8af0bc8a34962d106 (127.21.51.129). New cstate: current_term: 1 leader_uuid: "589753d403ab4cf8af0bc8a34962d106" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "589753d403ab4cf8af0bc8a34962d106" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 34749 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "30ad388f114948c38017799df2e20dc5" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 35859 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2e3534dcf6a842b5948550fd16a1b335" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 37709 } health_report { overall_health: UNKNOWN } } }
W20260809 16:20:22.207180 23297 tablet.cc:2369] T 0f4371da054442339e8e57da36f544d8 P 30ad388f114948c38017799df2e20dc5: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 09 16:20:22 dist-test-slave-s11t krb5kdc[22504](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292421, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.51.129@KRBTEST.COM
I20260809 16:20:22.223634 23325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:22.216012 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37536 (local address 127.21.51.129:34749)
0809 16:20:22.216077 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0809 16:20:22.216079 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:20:22.216175 (+ 96us) server_negotiation.cc:407] Connection header received
0809 16:20:22.216214 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:20:22.216215 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:20:22.216224 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:20:22.216278 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:20:22.216849 (+ 571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:22.217327 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:22.218001 (+ 674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:22.218193 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:22.218248 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:22.220659 (+ 2411us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:20:22.220671 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:20:22.220671 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:20:22.220678 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:20:22.222496 (+ 1818us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:20:22.222924 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:20:22.222927 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:20:22.222928 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:20:22.223003 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:20:22.223196 (+ 193us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:20:22.223196 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:20:22.223197 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:20:22.223342 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:20:22.223468 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0809 16:20:22.223499 (+ 31us) server_negotiation.cc:299] Negotiation successful
0809 16:20:22.223567 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
Aug 09 16:20:22 dist-test-slave-s11t krb5kdc[22504](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292422, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 09 16:20:22 dist-test-slave-s11t krb5kdc[22504](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292422, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.21.51.129@KRBTEST.COM
I20260809 16:20:22.248163 23325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:22.240625 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37552 (local address 127.21.51.129:34749)
0809 16:20:22.240734 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0809 16:20:22.240736 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:20:22.240847 (+ 111us) server_negotiation.cc:407] Connection header received
0809 16:20:22.240893 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:20:22.240894 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:20:22.240903 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:20:22.240956 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:20:22.241515 (+ 559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:22.241987 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:22.242731 (+ 744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:22.242916 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:22.242962 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:22.245532 (+ 2570us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:20:22.245533 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:20:22.245534 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:20:22.245541 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:20:22.246954 (+ 1413us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:20:22.247319 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:20:22.247320 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:20:22.247321 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:20:22.247379 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:20:22.247584 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:20:22.247585 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:20:22.247585 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:20:22.247685 (+ 100us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:20:22.247840 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0809 16:20:22.247976 (+ 136us) server_negotiation.cc:299] Negotiation successful
0809 16:20:22.248053 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
Aug 09 16:20:22 dist-test-slave-s11t krb5kdc[22504](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292422, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 09 16:20:22 dist-test-slave-s11t krb5kdc[22504](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292422, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.51.129@KRBTEST.COM
I20260809 16:20:22.274650 23325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:22.266162 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37564 (local address 127.21.51.129:34749)
0809 16:20:22.266253 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0809 16:20:22.266255 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:20:22.266350 (+ 95us) server_negotiation.cc:407] Connection header received
0809 16:20:22.266457 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:20:22.266458 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:20:22.266470 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:20:22.266554 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:20:22.267216 (+ 662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:22.267753 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:22.268578 (+ 825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:22.268956 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:22.269028 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:22.271713 (+ 2685us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:20:22.271714 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:20:22.271715 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:20:22.271722 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:20:22.273277 (+ 1555us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:20:22.273680 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:20:22.273682 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:20:22.273682 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:20:22.273805 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:20:22.274067 (+ 262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:20:22.274068 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:20:22.274068 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:20:22.274191 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:20:22.274291 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0809 16:20:22.274471 (+ 180us) server_negotiation.cc:299] Negotiation successful
0809 16:20:22.274539 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
W20260809 16:20:22.275202 22740 tablet_service.cc:2908] Rejecting scan request for tablet 0f4371da054442339e8e57da36f544d8: Uninitialized: clean time has not yet been initialized
I20260809 16:20:22.276204 21710 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrNqUCn/build/release/bin/kudu with pid 22678
I20260809 16:20:22.290264 21710 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrNqUCn/build/release/bin/kudu with pid 22896
I20260809 16:20:22.297206 21710 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrNqUCn/build/release/bin/kudu with pid 23103
I20260809 16:20:22.306588 21710 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrNqUCn/build/release/bin/kudu with pid 22521
2026-08-09T16:20:22Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (2406 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786292417722232-21710-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 16:20:22 dist-test-slave-s11t krb5kdc[23361](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 16:20:22 dist-test-slave-s11t krb5kdc[23361](info): set up 2 sockets
Aug 09 16:20:22 dist-test-slave-s11t krb5kdc[23361](info): commencing operation
krb5kdc: starting...
W20260809 16:20:23.551744 21710 mini_kdc.cc:121] Time spent starting KDC: real 1.221s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786292417722232-21710-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786292417722232-21710-0/krb5kdc/test-user.keytab.
Aug 09 16:20:23 dist-test-slave-s11t krb5kdc[23361](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292423, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T16:20:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T16:20:23Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.51.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.51.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.51.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.190.keytab.
Entry for principal kudu/127.21.51.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.190.keytab.
I20260809 16:20:23.676050 21710 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrNqUCn/build/release/bin/kudu
/tmp/dist-test-taskrNqUCn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786292417722232-21710-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786292417722232-21710-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786292417722232-21710-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786292417722232-21710-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.51.190:33935
--webserver_interface=127.21.51.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.190.keytab
--principal=kudu/127.21.51.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.51.148:39783
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.51.190:33935
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786292417722232-21710-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786292417722232-21710-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786292417722232-21710-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:20:23.758733 23385 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:20:23.758962 23385 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:20:23.759002 23385 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:20:23.760149 23385 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 16:20:23.760211 23385 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260809 16:20:23.760236 23385 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:20:23.760269 23385 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:20:23.760294 23385 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 16:20:23.760324 23385 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 16:20:23.761864 23385 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.51.148:39783
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786292417722232-21710-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786292417722232-21710-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.51.190:33935
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.190.keytab
--principal=kudu/127.21.51.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.51.190:33935
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786292417722232-21710-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.51.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23385
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786292417722232-21710-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:20:23.762136 23385 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:20:23.762360 23385 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:20:23.767302 23391 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:20:23.767302 23394 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:20:23.767328 23390 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:20:23.767673 23385 server_base.cc:1034] running on GCE node
I20260809 16:20:23.767944 23385 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:20:23.768256 23385 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:20:23.769436 23385 hybrid_clock.cc:648] HybridClock initialized: now 1786292423769423 us; error 48 us; skew 500 ppm
Aug 09 16:20:23 dist-test-slave-s11t krb5kdc[23361](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292423, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.51.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:20:23.776095 23385 init.cc:377] Logged in from keytab as kudu/127.21.51.190@KRBTEST.COM (short username kudu)
I20260809 16:20:23.776551 23385 webserver.cc:460] Webserver started at http://127.21.51.190:36863/ using document root <none> and password file <none>
I20260809 16:20:23.776820 23385 fs_manager.cc:359] Metadata directory not provided
I20260809 16:20:23.776870 23385 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:20:23.776963 23385 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:20:23.777956 23385 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786292417722232-21710-0/minicluster-data/master-0/data/instance:
uuid: "052d5e814a1d4882854b5c6f215a148b"
format_stamp: "Formatted at 2026-08-09 16:20:23 on dist-test-slave-s11t"
server_key: "a83e1e8124436138dae054b619da22ba"
server_key_iv: "9e6ee82b8a55a2bf3296aac225a0b9e7"
server_key_version: "encryptionkey@0"
I20260809 16:20:23.778266 23385 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786292417722232-21710-0/minicluster-data/master-0/wal/instance:
uuid: "052d5e814a1d4882854b5c6f215a148b"
format_stamp: "Formatted at 2026-08-09 16:20:23 on dist-test-slave-s11t"
server_key: "a83e1e8124436138dae054b619da22ba"
server_key_iv: "9e6ee82b8a55a2bf3296aac225a0b9e7"
server_key_version: "encryptionkey@0"
I20260809 16:20:23.779538 23385 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 16:20:23.780267 23409 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:20:23.780578 23385 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 16:20:23.780714 23385 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786292417722232-21710-0/minicluster-data/master-0/data,/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786292417722232-21710-0/minicluster-data/master-0/wal
uuid: "052d5e814a1d4882854b5c6f215a148b"
format_stamp: "Formatted at 2026-08-09 16:20:23 on dist-test-slave-s11t"
server_key: "a83e1e8124436138dae054b619da22ba"
server_key_iv: "9e6ee82b8a55a2bf3296aac225a0b9e7"
server_key_version: "encryptionkey@0"
I20260809 16:20:23.780826 23385 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786292417722232-21710-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786292417722232-21710-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786292417722232-21710-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:20:23.804193 23385 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:20:23.806320 23385 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:20:23.806490 23385 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:20:23.811269 23385 rpc_server.cc:307] RPC server started. Bound to: 127.21.51.190:33935
I20260809 16:20:23.811406 23512 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.51.190:33935 every 8 connection(s)
I20260809 16:20:23.811713 23385 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786292417722232-21710-0/minicluster-data/master-0/data/info.pb
I20260809 16:20:23.812434 23513 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:20:23.814973 23513 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 052d5e814a1d4882854b5c6f215a148b: Bootstrap starting.
I20260809 16:20:23.815621 23513 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 052d5e814a1d4882854b5c6f215a148b: Neither blocks nor log segments found. Creating new log.
I20260809 16:20:23.815954 23513 log.cc:824] T 00000000000000000000000000000000 P 052d5e814a1d4882854b5c6f215a148b: Log is configured to *not* fsync() on all Append() calls
I20260809 16:20:23.816776 23513 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 052d5e814a1d4882854b5c6f215a148b: No bootstrap required, opened a new log
I20260809 16:20:23.819144 23513 raft_consensus.cc:348] T 00000000000000000000000000000000 P 052d5e814a1d4882854b5c6f215a148b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "052d5e814a1d4882854b5c6f215a148b" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 33935 } }
I20260809 16:20:23.819291 23513 raft_consensus.cc:374] T 00000000000000000000000000000000 P 052d5e814a1d4882854b5c6f215a148b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:20:23.819343 23513 raft_consensus.cc:729] T 00000000000000000000000000000000 P 052d5e814a1d4882854b5c6f215a148b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 052d5e814a1d4882854b5c6f215a148b, State: Initialized, Role: FOLLOWER
I20260809 16:20:23.819537 23513 consensus_queue.cc:260] T 00000000000000000000000000000000 P 052d5e814a1d4882854b5c6f215a148b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "052d5e814a1d4882854b5c6f215a148b" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 33935 } }
I20260809 16:20:23.819633 23513 raft_consensus.cc:388] T 00000000000000000000000000000000 P 052d5e814a1d4882854b5c6f215a148b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 16:20:23.819692 23513 raft_consensus.cc:482] T 00000000000000000000000000000000 P 052d5e814a1d4882854b5c6f215a148b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 16:20:23.819748 23513 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 052d5e814a1d4882854b5c6f215a148b [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:20:23.820367 23513 raft_consensus.cc:504] T 00000000000000000000000000000000 P 052d5e814a1d4882854b5c6f215a148b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "052d5e814a1d4882854b5c6f215a148b" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 33935 } }
I20260809 16:20:23.820498 23513 leader_election.cc:302] T 00000000000000000000000000000000 P 052d5e814a1d4882854b5c6f215a148b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 052d5e814a1d4882854b5c6f215a148b; no voters:
I20260809 16:20:23.820745 23513 leader_election.cc:288] T 00000000000000000000000000000000 P 052d5e814a1d4882854b5c6f215a148b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 16:20:23.820780 23517 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 052d5e814a1d4882854b5c6f215a148b [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:20:23.820945 23517 raft_consensus.cc:686] T 00000000000000000000000000000000 P 052d5e814a1d4882854b5c6f215a148b [term 1 LEADER]: Becoming Leader. State: Replica: 052d5e814a1d4882854b5c6f215a148b, State: Running, Role: LEADER
I20260809 16:20:23.821036 23517 consensus_queue.cc:237] T 00000000000000000000000000000000 P 052d5e814a1d4882854b5c6f215a148b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "052d5e814a1d4882854b5c6f215a148b" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 33935 } }
I20260809 16:20:23.821039 23513 sys_catalog.cc:564] T 00000000000000000000000000000000 P 052d5e814a1d4882854b5c6f215a148b [sys.catalog]: configured and running, proceeding with master startup.
I20260809 16:20:23.821350 21710 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrNqUCn/build/release/bin/kudu as pid 23385
I20260809 16:20:23.821466 21710 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786292417722232-21710-0/minicluster-data/master-0/wal/instance
I20260809 16:20:23.821679 21710 external_mini_cluster.cc:1428] Setting key 821434ab0e694b12f0ca7e9c33f00890
I20260809 16:20:23.822229 23519 sys_catalog.cc:455] T 00000000000000000000000000000000 P 052d5e814a1d4882854b5c6f215a148b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 052d5e814a1d4882854b5c6f215a148b. Latest consensus state: current_term: 1 leader_uuid: "052d5e814a1d4882854b5c6f215a148b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "052d5e814a1d4882854b5c6f215a148b" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 33935 } } }
I20260809 16:20:23.822327 23519 sys_catalog.cc:458] T 00000000000000000000000000000000 P 052d5e814a1d4882854b5c6f215a148b [sys.catalog]: This master's current role is: LEADER
I20260809 16:20:23.822391 23518 sys_catalog.cc:455] T 00000000000000000000000000000000 P 052d5e814a1d4882854b5c6f215a148b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "052d5e814a1d4882854b5c6f215a148b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "052d5e814a1d4882854b5c6f215a148b" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 33935 } } }
I20260809 16:20:23.822511 23518 sys_catalog.cc:458] T 00000000000000000000000000000000 P 052d5e814a1d4882854b5c6f215a148b [sys.catalog]: This master's current role is: LEADER
W20260809 16:20:23.823151 23549 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 052d5e814a1d4882854b5c6f215a148b: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 16:20:23.823215 23549 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 16:20:23.823309 23544 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 16:20:23.823500 23544 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 16:20:23.825181 23544 catalog_manager.cc:1347] Generated new cluster ID: e7241884a0194cc988e81074e799290c
I20260809 16:20:23.825249 23544 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 09 16:20:23 dist-test-slave-s11t krb5kdc[23361](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292423, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.51.190@KRBTEST.COM
I20260809 16:20:23.834481 23542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:23.822283 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55060 (local address 127.21.51.190:33935)
0809 16:20:23.822925 (+ 642us) server_negotiation.cc:206] Beginning negotiation
0809 16:20:23.822945 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0809 16:20:23.822982 (+ 37us) server_negotiation.cc:407] Connection header received
0809 16:20:23.823110 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:20:23.823118 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:20:23.823179 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:20:23.823340 (+ 161us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:20:23.824223 (+ 883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:23.825655 (+ 1432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:23.826821 (+ 1166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:23.827448 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:23.827564 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:23.830532 (+ 2968us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:20:23.830536 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:20:23.830536 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:20:23.830548 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:20:23.832658 (+ 2110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:20:23.833122 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:20:23.833125 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:20:23.833125 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:20:23.833238 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:20:23.833467 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:20:23.833468 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:20:23.833469 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:20:23.833780 (+ 311us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:20:23.833943 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0809 16:20:23.834054 (+ 111us) server_negotiation.cc:299] Negotiation successful
0809 16:20:23.834209 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":586,"thread_start_us":104,"threads_started":1}
I20260809 16:20:23.841662 23544 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 16:20:23.842257 23544 catalog_manager.cc:1504] Loading token signing keys...
I20260809 16:20:23.854938 23544 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 052d5e814a1d4882854b5c6f215a148b: Generated new TSK 0
I20260809 16:20:23.855195 23544 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.21.51.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.51.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.51.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.129.keytab.
Entry for principal kudu/127.21.51.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.129.keytab.
I20260809 16:20:23.891271 21710 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrNqUCn/build/release/bin/kudu
/tmp/dist-test-taskrNqUCn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786292417722232-21710-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786292417722232-21710-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786292417722232-21710-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786292417722232-21710-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.51.129:0
--local_ip_for_outbound_sockets=127.21.51.129
--webserver_interface=127.21.51.129
--webserver_port=0
--tserver_master_addrs=127.21.51.190:33935
--keytab_file=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.129.keytab
--principal=kudu/127.21.51.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.51.148:39783
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786292417722232-21710-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786292417722232-21710-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786292417722232-21710-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:20:23.984767 23554 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:20:23.985045 23554 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:20:23.985083 23554 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:20:23.986383 23554 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:20:23.986430 23554 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:20:23.986451 23554 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 16:20:23.986497 23554 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.51.129
I20260809 16:20:23.988020 23554 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.51.148:39783
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786292417722232-21710-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786292417722232-21710-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.129.keytab
--principal=kudu/127.21.51.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.51.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786292417722232-21710-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.51.129
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.21.51.190:33935
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23554
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.51.129
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786292417722232-21710-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:20:23.988505 23554 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:20:23.988801 23554 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:20:23.993968 23563 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:20:23.993999 23561 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:20:23.994215 23554 server_base.cc:1034] running on GCE node
W20260809 16:20:23.993978 23560 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:20:23.994549 23554 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:20:23.994844 23554 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:20:23.995995 23554 hybrid_clock.cc:648] HybridClock initialized: now 1786292423995973 us; error 42 us; skew 500 ppm
Aug 09 16:20:23 dist-test-slave-s11t krb5kdc[23361](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292423, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.51.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:20:23.998622 23554 init.cc:377] Logged in from keytab as kudu/127.21.51.129@KRBTEST.COM (short username kudu)
I20260809 16:20:23.999003 23554 webserver.cc:460] Webserver started at http://127.21.51.129:39443/ using document root <none> and password file <none>
I20260809 16:20:23.999181 23554 fs_manager.cc:359] Metadata directory not provided
I20260809 16:20:23.999221 23554 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:20:23.999310 23554 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:20:24.000295 23554 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786292417722232-21710-0/minicluster-data/ts-0/data/instance:
uuid: "30b6ddb95a86440f92bac5652b288162"
format_stamp: "Formatted at 2026-08-09 16:20:23 on dist-test-slave-s11t"
server_key: "72dc40e1ad385c26637a2152ed4af64d"
server_key_iv: "96606b95f75939eec05f4921057c8869"
server_key_version: "encryptionkey@0"
I20260809 16:20:24.000604 23554 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786292417722232-21710-0/minicluster-data/ts-0/wal/instance:
uuid: "30b6ddb95a86440f92bac5652b288162"
format_stamp: "Formatted at 2026-08-09 16:20:23 on dist-test-slave-s11t"
server_key: "72dc40e1ad385c26637a2152ed4af64d"
server_key_iv: "96606b95f75939eec05f4921057c8869"
server_key_version: "encryptionkey@0"
I20260809 16:20:24.002048 23554 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260809 16:20:24.002836 23578 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:20:24.003100 23554 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 16:20:24.003238 23554 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786292417722232-21710-0/minicluster-data/ts-0/data,/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786292417722232-21710-0/minicluster-data/ts-0/wal
uuid: "30b6ddb95a86440f92bac5652b288162"
format_stamp: "Formatted at 2026-08-09 16:20:23 on dist-test-slave-s11t"
server_key: "72dc40e1ad385c26637a2152ed4af64d"
server_key_iv: "96606b95f75939eec05f4921057c8869"
server_key_version: "encryptionkey@0"
I20260809 16:20:24.003320 23554 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786292417722232-21710-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786292417722232-21710-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786292417722232-21710-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:20:24.019984 23554 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:20:24.022454 23554 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:20:24.022611 23554 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:20:24.023144 23554 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:20:24.023209 23554 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:20:24.023269 23554 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:20:24.023299 23554 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:20:24.030652 23554 rpc_server.cc:307] RPC server started. Bound to: 127.21.51.129:33979
I20260809 16:20:24.030745 23756 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.51.129:33979 every 8 connection(s)
I20260809 16:20:24.031072 23554 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786292417722232-21710-0/minicluster-data/ts-0/data/info.pb
Aug 09 16:20:24 dist-test-slave-s11t krb5kdc[23361](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292423, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.51.129@KRBTEST.COM for kudu/127.21.51.190@KRBTEST.COM
I20260809 16:20:24.034677 23542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:24.023734 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.51.129:45493 (local address 127.21.51.190:33935)
0809 16:20:24.023791 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0809 16:20:24.023794 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 16:20:24.024053 (+ 259us) server_negotiation.cc:407] Connection header received
0809 16:20:24.024655 (+ 602us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:20:24.024656 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:20:24.024672 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:20:24.024748 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:20:24.025717 (+ 969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:24.026692 (+ 975us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:24.028487 (+ 1795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:24.028847 (+ 360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:24.028955 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:24.031909 (+ 2954us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:20:24.031910 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:20:24.031910 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:20:24.031919 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:20:24.033217 (+ 1298us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:20:24.033626 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:20:24.033627 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:20:24.033627 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:20:24.033683 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:20:24.034011 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:20:24.034012 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:20:24.034013 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:20:24.034145 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:20:24.034211 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0809 16:20:24.034518 (+ 307us) server_negotiation.cc:299] Negotiation successful
0809 16:20:24.034606 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260809 16:20:24.034780 23593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:24.023718 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:33935 (local address 127.21.51.129:45493)
0809 16:20:24.023904 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0809 16:20:24.023953 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0809 16:20:24.024527 (+ 574us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:20:24.024849 (+ 322us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:20:24.024854 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:20:24.025254 (+ 400us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:20:24.025568 (+ 314us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:20:24.025576 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:20:24.026798 (+ 1222us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:20:24.026799 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:20:24.028393 (+ 1594us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:20:24.028397 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:20:24.028958 (+ 561us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:20:24.028959 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:20:24.029089 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:24.029653 (+ 564us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:20:24.029665 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:20:24.031810 (+ 2145us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:20:24.033306 (+ 1496us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:20:24.033312 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:20:24.033314 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:20:24.033550 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:20:24.033791 (+ 241us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:20:24.033792 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:20:24.033792 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:20:24.033908 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:20:24.034227 (+ 319us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:20:24.034232 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:20:24.034370 (+ 138us) client_negotiation.cc:769] Sending connection context
0809 16:20:24.034456 (+ 86us) client_negotiation.cc:240] Negotiation successful
0809 16:20:24.034582 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":90,"threads_started":1}
I20260809 16:20:24.035323 23758 heartbeater.cc:347] Connected to a master server at 127.21.51.190:33935
I20260809 16:20:24.035421 23758 heartbeater.cc:464] Registering TS with master...
I20260809 16:20:24.035609 23758 heartbeater.cc:511] Master 127.21.51.190:33935 requested a full tablet report, sending...
I20260809 16:20:24.036159 23436 ts_manager.cc:194] Registered new tserver with Master: 30b6ddb95a86440f92bac5652b288162 (127.21.51.129:33979)
I20260809 16:20:24.037168 23436 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.51.129@KRBTEST.COM'} at 127.21.51.129:45493
I20260809 16:20:24.037483 21710 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrNqUCn/build/release/bin/kudu as pid 23554
I20260809 16:20:24.037592 21710 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786292417722232-21710-0/minicluster-data/ts-0/wal/instance
I20260809 16:20:24.037755 21710 external_mini_cluster.cc:1428] Setting key 58f66acb8712760c49500b78c760dc67
I20260809 16:20:24.043015 23542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:24.036419 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.51.129:48053 (local address 127.21.51.190:33935)
0809 16:20:24.036463 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0809 16:20:24.036465 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:20:24.036570 (+ 105us) server_negotiation.cc:407] Connection header received
0809 16:20:24.036621 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:20:24.036622 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:20:24.036630 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:20:24.036676 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:20:24.037069 (+ 393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:24.037718 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:24.038842 (+ 1124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:24.039030 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:24.039087 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:24.040127 (+ 1040us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:20:24.040128 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:20:24.040128 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:20:24.040134 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:20:24.041821 (+ 1687us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:20:24.042175 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:20:24.042176 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:20:24.042177 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:20:24.042241 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:20:24.042468 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:20:24.042469 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:20:24.042469 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:20:24.042588 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:20:24.042660 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0809 16:20:24.042841 (+ 181us) server_negotiation.cc:299] Negotiation successful
0809 16:20:24.042920 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260809 16:20:24.043026 23773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:24.036341 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:33935 (local address 127.21.51.129:48053)
0809 16:20:24.036492 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0809 16:20:24.036504 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 16:20:24.036566 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:20:24.036680 (+ 114us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:20:24.036699 (+ 19us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:20:24.036939 (+ 240us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:20:24.036997 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:20:24.036999 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:20:24.037957 (+ 958us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:20:24.037959 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:20:24.038753 (+ 794us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:20:24.038757 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:20:24.039105 (+ 348us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:20:24.039107 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:20:24.039192 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:24.039602 (+ 410us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:20:24.039607 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:20:24.040041 (+ 434us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:20:24.041899 (+ 1858us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:20:24.041900 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:20:24.041900 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:20:24.042066 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:20:24.042320 (+ 254us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:20:24.042321 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:20:24.042322 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:20:24.042396 (+ 74us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:20:24.042664 (+ 268us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:20:24.042665 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:20:24.042727 (+ 62us) client_negotiation.cc:769] Sending connection context
0809 16:20:24.042812 (+ 85us) client_negotiation.cc:240] Negotiation successful
0809 16:20:24.042920 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":94,"threads_started":1}
WARNING: no policy specified for kudu/127.21.51.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.51.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.51.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.130.keytab.
Entry for principal kudu/127.21.51.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.130.keytab.
I20260809 16:20:24.065589 21710 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrNqUCn/build/release/bin/kudu
/tmp/dist-test-taskrNqUCn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786292417722232-21710-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786292417722232-21710-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786292417722232-21710-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786292417722232-21710-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.51.130:0
--local_ip_for_outbound_sockets=127.21.51.130
--webserver_interface=127.21.51.130
--webserver_port=0
--tserver_master_addrs=127.21.51.190:33935
--keytab_file=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.130.keytab
--principal=kudu/127.21.51.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.51.148:39783
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786292417722232-21710-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786292417722232-21710-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786292417722232-21710-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:20:24.149035 23777 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:20:24.149238 23777 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:20:24.149262 23777 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:20:24.150426 23777 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:20:24.150471 23777 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:20:24.150492 23777 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 16:20:24.150509 23777 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.51.130
I20260809 16:20:24.152022 23777 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.51.148:39783
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786292417722232-21710-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786292417722232-21710-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.130.keytab
--principal=kudu/127.21.51.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.51.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786292417722232-21710-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.51.130
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.21.51.190:33935
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23777
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.51.130
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786292417722232-21710-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:20:24.152275 23777 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:20:24.152503 23777 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:20:24.157632 23788 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:20:24.157667 23784 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:20:24.157655 23790 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:20:24.157922 23777 server_base.cc:1034] running on GCE node
I20260809 16:20:24.158145 23777 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:20:24.158406 23777 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:20:24.159556 23777 hybrid_clock.cc:648] HybridClock initialized: now 1786292424159535 us; error 38 us; skew 500 ppm
Aug 09 16:20:24 dist-test-slave-s11t krb5kdc[23361](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292424, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.51.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:20:24.162139 23777 init.cc:377] Logged in from keytab as kudu/127.21.51.130@KRBTEST.COM (short username kudu)
I20260809 16:20:24.162643 23777 webserver.cc:460] Webserver started at http://127.21.51.130:43313/ using document root <none> and password file <none>
I20260809 16:20:24.162922 23777 fs_manager.cc:359] Metadata directory not provided
I20260809 16:20:24.162972 23777 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:20:24.163147 23777 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:20:24.164283 23777 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786292417722232-21710-0/minicluster-data/ts-1/data/instance:
uuid: "3b728a70ee654948ab2fdf9f6d290630"
format_stamp: "Formatted at 2026-08-09 16:20:24 on dist-test-slave-s11t"
server_key: "b4c6105ba5de472b44af1126679a1374"
server_key_iv: "ad363c72baa02bfd55d2a446765f3e77"
server_key_version: "encryptionkey@0"
I20260809 16:20:24.164664 23777 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786292417722232-21710-0/minicluster-data/ts-1/wal/instance:
uuid: "3b728a70ee654948ab2fdf9f6d290630"
format_stamp: "Formatted at 2026-08-09 16:20:24 on dist-test-slave-s11t"
server_key: "b4c6105ba5de472b44af1126679a1374"
server_key_iv: "ad363c72baa02bfd55d2a446765f3e77"
server_key_version: "encryptionkey@0"
I20260809 16:20:24.166108 23777 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:20:24.166939 23799 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:20:24.167203 23777 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:20:24.167325 23777 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786292417722232-21710-0/minicluster-data/ts-1/data,/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786292417722232-21710-0/minicluster-data/ts-1/wal
uuid: "3b728a70ee654948ab2fdf9f6d290630"
format_stamp: "Formatted at 2026-08-09 16:20:24 on dist-test-slave-s11t"
server_key: "b4c6105ba5de472b44af1126679a1374"
server_key_iv: "ad363c72baa02bfd55d2a446765f3e77"
server_key_version: "encryptionkey@0"
I20260809 16:20:24.167398 23777 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786292417722232-21710-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786292417722232-21710-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786292417722232-21710-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:20:24.180998 23777 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:20:24.182868 23777 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:20:24.183023 23777 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:20:24.183661 23777 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:20:24.183722 23777 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:20:24.183753 23777 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:20:24.183771 23777 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 16:20:24 dist-test-slave-s11t krb5kdc[23361](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292424, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.51.130@KRBTEST.COM for kudu/127.21.51.190@KRBTEST.COM
I20260809 16:20:24.191306 23777 rpc_server.cc:307] RPC server started. Bound to: 127.21.51.130:41839
I20260809 16:20:24.191442 23977 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.51.130:41839 every 8 connection(s)
I20260809 16:20:24.191715 23777 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786292417722232-21710-0/minicluster-data/ts-1/data/info.pb
I20260809 16:20:24.193307 23542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:24.184015 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.51.130:35259 (local address 127.21.51.190:33935)
0809 16:20:24.184074 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0809 16:20:24.184076 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:20:24.184319 (+ 243us) server_negotiation.cc:407] Connection header received
0809 16:20:24.184938 (+ 619us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:20:24.184939 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:20:24.184954 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:20:24.185011 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:20:24.185673 (+ 662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:24.186186 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:24.187211 (+ 1025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:24.187494 (+ 283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:24.187572 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:24.190045 (+ 2473us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:20:24.190046 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:20:24.190046 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:20:24.190054 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:20:24.191731 (+ 1677us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:20:24.192226 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:20:24.192228 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:20:24.192228 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:20:24.192289 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:20:24.192572 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:20:24.192573 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:20:24.192573 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:20:24.192741 (+ 168us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:20:24.192831 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0809 16:20:24.193123 (+ 292us) server_negotiation.cc:299] Negotiation successful
0809 16:20:24.193217 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260809 16:20:24.193478 23811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:24.184000 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:33935 (local address 127.21.51.130:35259)
0809 16:20:24.184193 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0809 16:20:24.184228 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0809 16:20:24.184799 (+ 571us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:20:24.185084 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:20:24.185088 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:20:24.185391 (+ 303us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:20:24.185574 (+ 183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:20:24.185579 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:20:24.186285 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:20:24.186287 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:20:24.187112 (+ 825us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:20:24.187114 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:20:24.187578 (+ 464us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:20:24.187579 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:20:24.187748 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:24.188269 (+ 521us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:20:24.188279 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:20:24.189937 (+ 1658us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:20:24.191826 (+ 1889us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:20:24.191831 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:20:24.191833 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:20:24.192157 (+ 324us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:20:24.192382 (+ 225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:20:24.192382 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:20:24.192383 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:20:24.192494 (+ 111us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:20:24.192834 (+ 340us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:20:24.192841 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:20:24.192978 (+ 137us) client_negotiation.cc:769] Sending connection context
0809 16:20:24.193083 (+ 105us) client_negotiation.cc:240] Negotiation successful
0809 16:20:24.193241 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":105,"threads_started":1}
I20260809 16:20:24.194003 23978 heartbeater.cc:347] Connected to a master server at 127.21.51.190:33935
I20260809 16:20:24.194087 23978 heartbeater.cc:464] Registering TS with master...
I20260809 16:20:24.194271 23978 heartbeater.cc:511] Master 127.21.51.190:33935 requested a full tablet report, sending...
I20260809 16:20:24.194676 23435 ts_manager.cc:194] Registered new tserver with Master: 3b728a70ee654948ab2fdf9f6d290630 (127.21.51.130:41839)
I20260809 16:20:24.195215 23435 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.51.130@KRBTEST.COM'} at 127.21.51.130:35259
I20260809 16:20:24.200798 23542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:24.195131 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.51.130:37069 (local address 127.21.51.190:33935)
0809 16:20:24.195168 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0809 16:20:24.195169 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 16:20:24.195231 (+ 62us) server_negotiation.cc:407] Connection header received
0809 16:20:24.195329 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:20:24.195330 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:20:24.195341 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:20:24.195405 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:20:24.195725 (+ 320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:24.196218 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:24.196995 (+ 777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:24.197203 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:24.197296 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:24.198199 (+ 903us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:20:24.198200 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:20:24.198200 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:20:24.198207 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:20:24.199595 (+ 1388us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:20:24.199956 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:20:24.199957 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:20:24.199958 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:20:24.200013 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:20:24.200229 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:20:24.200230 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:20:24.200230 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:20:24.200382 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:20:24.200459 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0809 16:20:24.200621 (+ 162us) server_negotiation.cc:299] Negotiation successful
0809 16:20:24.200711 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260809 16:20:24.200816 23986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:24.195037 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:33935 (local address 127.21.51.130:37069)
0809 16:20:24.195171 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0809 16:20:24.195182 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 16:20:24.195245 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:20:24.195411 (+ 166us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:20:24.195412 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:20:24.195587 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:20:24.195635 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:20:24.195636 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:20:24.196326 (+ 690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:20:24.196327 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:20:24.196914 (+ 587us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:20:24.196916 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:20:24.197297 (+ 381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:20:24.197298 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:20:24.197382 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:24.197808 (+ 426us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:20:24.197813 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:20:24.198089 (+ 276us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:20:24.199700 (+ 1611us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:20:24.199701 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:20:24.199702 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:20:24.199881 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:20:24.200094 (+ 213us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:20:24.200095 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:20:24.200096 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:20:24.200161 (+ 65us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:20:24.200462 (+ 301us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:20:24.200463 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:20:24.200508 (+ 45us) client_negotiation.cc:769] Sending connection context
0809 16:20:24.200595 (+ 87us) client_negotiation.cc:240] Negotiation successful
0809 16:20:24.200716 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":85,"threads_started":1}
I20260809 16:20:24.201154 21710 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrNqUCn/build/release/bin/kudu as pid 23777
I20260809 16:20:24.201270 21710 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786292417722232-21710-0/minicluster-data/ts-1/wal/instance
I20260809 16:20:24.201444 21710 external_mini_cluster.cc:1428] Setting key 9eec3a718ff46d016e853b0c4db0395e
WARNING: no policy specified for kudu/127.21.51.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.51.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.51.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.131.keytab.
Entry for principal kudu/127.21.51.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.131.keytab.
I20260809 16:20:24.225602 21710 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrNqUCn/build/release/bin/kudu
/tmp/dist-test-taskrNqUCn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786292417722232-21710-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786292417722232-21710-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786292417722232-21710-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786292417722232-21710-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.51.131:0
--local_ip_for_outbound_sockets=127.21.51.131
--webserver_interface=127.21.51.131
--webserver_port=0
--tserver_master_addrs=127.21.51.190:33935
--keytab_file=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.131.keytab
--principal=kudu/127.21.51.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.51.148:39783
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786292417722232-21710-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786292417722232-21710-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786292417722232-21710-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:20:24.310588 23990 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:20:24.310776 23990 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:20:24.310799 23990 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:20:24.311947 23990 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:20:24.311990 23990 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:20:24.312011 23990 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 16:20:24.312073 23990 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.51.131
I20260809 16:20:24.313576 23990 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.51.148:39783
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786292417722232-21710-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786292417722232-21710-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.131.keytab
--principal=kudu/127.21.51.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.51.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786292417722232-21710-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.51.131
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.21.51.190:33935
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23990
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.51.131
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786292417722232-21710-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:20:24.313817 23990 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:20:24.314050 23990 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 16:20:24.319134 23990 server_base.cc:1034] running on GCE node
W20260809 16:20:24.319108 24001 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:20:24.319147 24003 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:20:24.319116 24007 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:20:24.319588 23990 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:20:24.319878 23990 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:20:24.321008 23990 hybrid_clock.cc:648] HybridClock initialized: now 1786292424320983 us; error 27 us; skew 500 ppm
Aug 09 16:20:24 dist-test-slave-s11t krb5kdc[23361](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292424, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.51.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:20:24.323616 23990 init.cc:377] Logged in from keytab as kudu/127.21.51.131@KRBTEST.COM (short username kudu)
I20260809 16:20:24.324030 23990 webserver.cc:460] Webserver started at http://127.21.51.131:39759/ using document root <none> and password file <none>
I20260809 16:20:24.324263 23990 fs_manager.cc:359] Metadata directory not provided
I20260809 16:20:24.324339 23990 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:20:24.324496 23990 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:20:24.325613 23990 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786292417722232-21710-0/minicluster-data/ts-2/data/instance:
uuid: "c241eb2dc27b4e96a82c4b3feade2d7f"
format_stamp: "Formatted at 2026-08-09 16:20:24 on dist-test-slave-s11t"
server_key: "7ed9b81fa76d41cb1f6a3284edbe5c20"
server_key_iv: "478856d3fc77c1844bcf9554b640bea5"
server_key_version: "encryptionkey@0"
I20260809 16:20:24.325984 23990 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786292417722232-21710-0/minicluster-data/ts-2/wal/instance:
uuid: "c241eb2dc27b4e96a82c4b3feade2d7f"
format_stamp: "Formatted at 2026-08-09 16:20:24 on dist-test-slave-s11t"
server_key: "7ed9b81fa76d41cb1f6a3284edbe5c20"
server_key_iv: "478856d3fc77c1844bcf9554b640bea5"
server_key_version: "encryptionkey@0"
I20260809 16:20:24.327298 23990 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 16:20:24.327996 24020 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:20:24.328172 23990 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 16:20:24.328274 23990 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786292417722232-21710-0/minicluster-data/ts-2/data,/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786292417722232-21710-0/minicluster-data/ts-2/wal
uuid: "c241eb2dc27b4e96a82c4b3feade2d7f"
format_stamp: "Formatted at 2026-08-09 16:20:24 on dist-test-slave-s11t"
server_key: "7ed9b81fa76d41cb1f6a3284edbe5c20"
server_key_iv: "478856d3fc77c1844bcf9554b640bea5"
server_key_version: "encryptionkey@0"
I20260809 16:20:24.328373 23990 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786292417722232-21710-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786292417722232-21710-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786292417722232-21710-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:20:24.339788 23990 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:20:24.342094 23990 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:20:24.342255 23990 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:20:24.342752 23990 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:20:24.342818 23990 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:20:24.342854 23990 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:20:24.342900 23990 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 16:20:24 dist-test-slave-s11t krb5kdc[23361](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292424, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.51.131@KRBTEST.COM for kudu/127.21.51.190@KRBTEST.COM
I20260809 16:20:24.350204 23990 rpc_server.cc:307] RPC server started. Bound to: 127.21.51.131:43627
I20260809 16:20:24.350252 24191 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.51.131:43627 every 8 connection(s)
I20260809 16:20:24.350610 23990 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786292417722232-21710-0/minicluster-data/ts-2/data/info.pb
I20260809 16:20:24.353194 23542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:24.343278 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.51.131:40655 (local address 127.21.51.190:33935)
0809 16:20:24.343353 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0809 16:20:24.343356 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 16:20:24.343511 (+ 155us) server_negotiation.cc:407] Connection header received
0809 16:20:24.343972 (+ 461us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:20:24.343974 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:20:24.343989 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:20:24.344074 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:20:24.344933 (+ 859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:24.345447 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:24.346461 (+ 1014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:24.346780 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:24.346855 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:24.350529 (+ 3674us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:20:24.350530 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:20:24.350531 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:20:24.350539 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:20:24.351948 (+ 1409us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:20:24.352336 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:20:24.352337 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:20:24.352337 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:20:24.352394 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:20:24.352607 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:20:24.352608 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:20:24.352608 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:20:24.352776 (+ 168us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:20:24.352855 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0809 16:20:24.353062 (+ 207us) server_negotiation.cc:299] Negotiation successful
0809 16:20:24.353131 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260809 16:20:24.353403 24031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:24.343229 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:33935 (local address 127.21.51.131:40655)
0809 16:20:24.343383 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0809 16:20:24.343422 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0809 16:20:24.343870 (+ 448us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:20:24.344119 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:20:24.344123 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:20:24.344499 (+ 376us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:20:24.344825 (+ 326us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:20:24.344831 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:20:24.345554 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:20:24.345555 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:20:24.346365 (+ 810us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:20:24.346367 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:20:24.346881 (+ 514us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:20:24.346883 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:20:24.347068 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:24.347608 (+ 540us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:20:24.347620 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:20:24.350403 (+ 2783us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:20:24.352023 (+ 1620us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:20:24.352029 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:20:24.352031 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:20:24.352251 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:20:24.352455 (+ 204us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:20:24.352455 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:20:24.352455 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:20:24.352551 (+ 96us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:20:24.352849 (+ 298us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:20:24.352852 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:20:24.352965 (+ 113us) client_negotiation.cc:769] Sending connection context
0809 16:20:24.353045 (+ 80us) client_negotiation.cc:240] Negotiation successful
0809 16:20:24.353164 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":86,"threads_started":1}
I20260809 16:20:24.353963 24192 heartbeater.cc:347] Connected to a master server at 127.21.51.190:33935
I20260809 16:20:24.354071 24192 heartbeater.cc:464] Registering TS with master...
I20260809 16:20:24.354259 24192 heartbeater.cc:511] Master 127.21.51.190:33935 requested a full tablet report, sending...
I20260809 16:20:24.354702 23436 ts_manager.cc:194] Registered new tserver with Master: c241eb2dc27b4e96a82c4b3feade2d7f (127.21.51.131:43627)
I20260809 16:20:24.355208 23436 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.51.131@KRBTEST.COM'} at 127.21.51.131:40655
I20260809 16:20:24.360525 23542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:24.355610 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.51.131:55035 (local address 127.21.51.190:33935)
0809 16:20:24.355663 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0809 16:20:24.355665 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:20:24.355675 (+ 10us) server_negotiation.cc:407] Connection header received
0809 16:20:24.355703 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:20:24.355704 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:20:24.355715 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:20:24.355775 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:20:24.356088 (+ 313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:24.356563 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:24.357285 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:24.357466 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:24.357519 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:24.358262 (+ 743us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:20:24.358263 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:20:24.358263 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:20:24.358269 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:20:24.359707 (+ 1438us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:20:24.359971 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:20:24.359972 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:20:24.359972 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:20:24.360050 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:20:24.360191 (+ 141us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:20:24.360191 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:20:24.360191 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:20:24.360272 (+ 81us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:20:24.360316 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0809 16:20:24.360416 (+ 100us) server_negotiation.cc:299] Negotiation successful
0809 16:20:24.360470 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260809 16:20:24.360538 24202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:24.355472 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:33935 (local address 127.21.51.131:55035)
0809 16:20:24.355614 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0809 16:20:24.355624 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 16:20:24.355668 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:20:24.355782 (+ 114us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:20:24.355783 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:20:24.355948 (+ 165us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:20:24.356010 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:20:24.356011 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:20:24.356644 (+ 633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:20:24.356645 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:20:24.357213 (+ 568us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:20:24.357215 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:20:24.357532 (+ 317us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:20:24.357533 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:20:24.357590 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:24.357925 (+ 335us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:20:24.357929 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:20:24.358180 (+ 251us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:20:24.359774 (+ 1594us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:20:24.359774 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:20:24.359775 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:20:24.359927 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:20:24.360105 (+ 178us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:20:24.360106 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:20:24.360106 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:20:24.360154 (+ 48us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:20:24.360318 (+ 164us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:20:24.360319 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:20:24.360352 (+ 33us) client_negotiation.cc:769] Sending connection context
0809 16:20:24.360401 (+ 49us) client_negotiation.cc:240] Negotiation successful
0809 16:20:24.360473 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":88,"threads_started":1}
I20260809 16:20:24.360839 21710 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrNqUCn/build/release/bin/kudu as pid 23990
I20260809 16:20:24.360934 21710 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786292417722232-21710-0/minicluster-data/ts-2/wal/instance
I20260809 16:20:24.361123 21710 external_mini_cluster.cc:1428] Setting key 54f392358d476be1354018aec794760a
I20260809 16:20:24.362319 21710 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 09 16:20:24 dist-test-slave-s11t krb5kdc[23361](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292424, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 09 16:20:24 dist-test-slave-s11t krb5kdc[23361](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292424, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.21.51.190@KRBTEST.COM
I20260809 16:20:24.387243 23542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:24.380135 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55066 (local address 127.21.51.190:33935)
0809 16:20:24.380202 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0809 16:20:24.380204 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:20:24.380217 (+ 13us) server_negotiation.cc:407] Connection header received
0809 16:20:24.380240 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:20:24.380240 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:20:24.380248 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:20:24.380308 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:20:24.380951 (+ 643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:24.381450 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:24.382190 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:24.382390 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:24.382469 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:24.384841 (+ 2372us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:20:24.384842 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:20:24.384843 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:20:24.384850 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:20:24.386074 (+ 1224us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:20:24.386445 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:20:24.386446 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:20:24.386447 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:20:24.386508 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:20:24.386719 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:20:24.386720 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:20:24.386720 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:20:24.386829 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:20:24.386957 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0809 16:20:24.387057 (+ 100us) server_negotiation.cc:299] Negotiation successful
0809 16:20:24.387140 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260809 16:20:24.388823 23436 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55066:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 16:20:24.390342 23436 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 16:20:24.397264 24217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:24.392833 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.130:41839 (local address 127.0.0.1:37706)
0809 16:20:24.393054 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0809 16:20:24.393063 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 16:20:24.393118 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:20:24.393715 (+ 597us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:20:24.393716 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:20:24.393719 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:20:24.393774 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:20:24.393775 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:20:24.394736 (+ 961us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:20:24.394737 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:20:24.396158 (+ 1421us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:20:24.396161 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:20:24.397022 (+ 861us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:20:24.397023 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:20:24.397083 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:24.397086 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 16:20:24.397139 (+ 53us) client_negotiation.cc:240] Negotiation successful
0809 16:20:24.397172 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":211,"mutex_wait_us":68,"thread_start_us":66,"threads_started":1}
I20260809 16:20:24.397285 24226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:24.393247 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37706 (local address 127.21.51.130:41839)
0809 16:20:24.393534 (+ 287us) server_negotiation.cc:206] Beginning negotiation
0809 16:20:24.393539 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 16:20:24.393548 (+ 9us) server_negotiation.cc:407] Connection header received
0809 16:20:24.393587 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:20:24.393589 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:20:24.393605 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:20:24.393713 (+ 108us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:20:24.393831 (+ 118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:24.394648 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:24.396236 (+ 1588us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:24.396956 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:24.397025 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:24.397079 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0809 16:20:24.397150 (+ 71us) server_negotiation.cc:299] Negotiation successful
0809 16:20:24.397185 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":252,"thread_start_us":132,"threads_started":1}
I20260809 16:20:24.397363 24220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:24.392885 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48936 (local address 127.21.51.129:33979)
0809 16:20:24.393272 (+ 387us) server_negotiation.cc:206] Beginning negotiation
0809 16:20:24.393277 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 16:20:24.393288 (+ 11us) server_negotiation.cc:407] Connection header received
0809 16:20:24.393415 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:20:24.393417 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:20:24.393434 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:20:24.393514 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:20:24.393773 (+ 259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:24.394964 (+ 1191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:24.396293 (+ 1329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:24.396821 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:24.396902 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:24.396964 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0809 16:20:24.397215 (+ 251us) server_negotiation.cc:299] Negotiation successful
0809 16:20:24.397256 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":346,"thread_start_us":109,"threads_started":1}
I20260809 16:20:24.397603 24223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:24.392881 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47068 (local address 127.21.51.131:43627)
0809 16:20:24.393466 (+ 585us) server_negotiation.cc:206] Beginning negotiation
0809 16:20:24.393470 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 16:20:24.393481 (+ 11us) server_negotiation.cc:407] Connection header received
0809 16:20:24.393517 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:20:24.393518 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:20:24.393532 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:20:24.393777 (+ 245us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:20:24.393834 (+ 57us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:24.394931 (+ 1097us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:24.396144 (+ 1213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:24.396918 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:24.397321 (+ 403us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:24.397390 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0809 16:20:24.397417 (+ 27us) server_negotiation.cc:299] Negotiation successful
0809 16:20:24.397464 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":548,"thread_start_us":300,"threads_started":1}
I20260809 16:20:24.397276 24218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:24.392832 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.131:43627 (local address 127.0.0.1:47068)
0809 16:20:24.393007 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0809 16:20:24.393050 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0809 16:20:24.393118 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:20:24.393626 (+ 508us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:20:24.393627 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:20:24.393629 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:20:24.393700 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:20:24.393702 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:20:24.395048 (+ 1346us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:20:24.395050 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:20:24.396047 (+ 997us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:20:24.396049 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:20:24.397031 (+ 982us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:20:24.397032 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:20:24.397098 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:24.397102 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 16:20:24.397135 (+ 33us) client_negotiation.cc:240] Negotiation successful
0809 16:20:24.397183 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"mutex_wait_us":95,"thread_start_us":71,"threads_started":1}
I20260809 16:20:24.398052 24225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:24.393121 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.129:33979 (local address 127.0.0.1:48936)
0809 16:20:24.393234 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0809 16:20:24.393244 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 16:20:24.393310 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:20:24.393608 (+ 298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:20:24.393611 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:20:24.393613 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:20:24.393685 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:20:24.393688 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:20:24.395062 (+ 1374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:20:24.395063 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:20:24.396224 (+ 1161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:20:24.396226 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:20:24.396915 (+ 689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:20:24.396916 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:20:24.397009 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:24.397024 (+ 15us) client_negotiation.cc:769] Sending connection context
0809 16:20:24.397090 (+ 66us) client_negotiation.cc:240] Negotiation successful
0809 16:20:24.397150 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":99,"mutex_wait_us":2,"thread_start_us":79,"threads_started":1}
I20260809 16:20:24.398172 23875 tablet_service.cc:1467] Processing CreateTablet for tablet 7479a6ee556a4056a0ce4d5a1ba4b323 (DEFAULT_TABLE table=test-table [id=95d260c896ff42159411523b140294d3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:20:24.398571 23875 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7479a6ee556a4056a0ce4d5a1ba4b323. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:20:24.398690 23669 tablet_service.cc:1467] Processing CreateTablet for tablet 7479a6ee556a4056a0ce4d5a1ba4b323 (DEFAULT_TABLE table=test-table [id=95d260c896ff42159411523b140294d3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:20:24.399022 23669 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7479a6ee556a4056a0ce4d5a1ba4b323. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:20:24.401678 24227 tablet_bootstrap.cc:492] T 7479a6ee556a4056a0ce4d5a1ba4b323 P 3b728a70ee654948ab2fdf9f6d290630: Bootstrap starting.
I20260809 16:20:24.401954 24231 tablet_bootstrap.cc:492] T 7479a6ee556a4056a0ce4d5a1ba4b323 P 30b6ddb95a86440f92bac5652b288162: Bootstrap starting.
I20260809 16:20:24.402024 24099 tablet_service.cc:1467] Processing CreateTablet for tablet 7479a6ee556a4056a0ce4d5a1ba4b323 (DEFAULT_TABLE table=test-table [id=95d260c896ff42159411523b140294d3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:20:24.402369 24099 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7479a6ee556a4056a0ce4d5a1ba4b323. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:20:24.402496 24227 tablet_bootstrap.cc:654] T 7479a6ee556a4056a0ce4d5a1ba4b323 P 3b728a70ee654948ab2fdf9f6d290630: Neither blocks nor log segments found. Creating new log.
I20260809 16:20:24.402602 24231 tablet_bootstrap.cc:654] T 7479a6ee556a4056a0ce4d5a1ba4b323 P 30b6ddb95a86440f92bac5652b288162: Neither blocks nor log segments found. Creating new log.
I20260809 16:20:24.402854 24227 log.cc:824] T 7479a6ee556a4056a0ce4d5a1ba4b323 P 3b728a70ee654948ab2fdf9f6d290630: Log is configured to *not* fsync() on all Append() calls
I20260809 16:20:24.402966 24231 log.cc:824] T 7479a6ee556a4056a0ce4d5a1ba4b323 P 30b6ddb95a86440f92bac5652b288162: Log is configured to *not* fsync() on all Append() calls
I20260809 16:20:24.403666 24227 tablet_bootstrap.cc:492] T 7479a6ee556a4056a0ce4d5a1ba4b323 P 3b728a70ee654948ab2fdf9f6d290630: No bootstrap required, opened a new log
I20260809 16:20:24.403677 24231 tablet_bootstrap.cc:492] T 7479a6ee556a4056a0ce4d5a1ba4b323 P 30b6ddb95a86440f92bac5652b288162: No bootstrap required, opened a new log
I20260809 16:20:24.403745 24227 ts_tablet_manager.cc:1397] T 7479a6ee556a4056a0ce4d5a1ba4b323 P 3b728a70ee654948ab2fdf9f6d290630: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 16:20:24.403762 24231 ts_tablet_manager.cc:1397] T 7479a6ee556a4056a0ce4d5a1ba4b323 P 30b6ddb95a86440f92bac5652b288162: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 16:20:24.405408 24231 raft_consensus.cc:348] T 7479a6ee556a4056a0ce4d5a1ba4b323 P 30b6ddb95a86440f92bac5652b288162 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c241eb2dc27b4e96a82c4b3feade2d7f" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 43627 } } peers { permanent_uuid: "3b728a70ee654948ab2fdf9f6d290630" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 41839 } } peers { permanent_uuid: "30b6ddb95a86440f92bac5652b288162" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 33979 } }
I20260809 16:20:24.405601 24231 raft_consensus.cc:374] T 7479a6ee556a4056a0ce4d5a1ba4b323 P 30b6ddb95a86440f92bac5652b288162 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:20:24.405649 24231 raft_consensus.cc:729] T 7479a6ee556a4056a0ce4d5a1ba4b323 P 30b6ddb95a86440f92bac5652b288162 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 30b6ddb95a86440f92bac5652b288162, State: Initialized, Role: FOLLOWER
I20260809 16:20:24.405756 24231 consensus_queue.cc:260] T 7479a6ee556a4056a0ce4d5a1ba4b323 P 30b6ddb95a86440f92bac5652b288162 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c241eb2dc27b4e96a82c4b3feade2d7f" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 43627 } } peers { permanent_uuid: "3b728a70ee654948ab2fdf9f6d290630" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 41839 } } peers { permanent_uuid: "30b6ddb95a86440f92bac5652b288162" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 33979 } }
I20260809 16:20:24.405804 24227 raft_consensus.cc:348] T 7479a6ee556a4056a0ce4d5a1ba4b323 P 3b728a70ee654948ab2fdf9f6d290630 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c241eb2dc27b4e96a82c4b3feade2d7f" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 43627 } } peers { permanent_uuid: "3b728a70ee654948ab2fdf9f6d290630" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 41839 } } peers { permanent_uuid: "30b6ddb95a86440f92bac5652b288162" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 33979 } }
I20260809 16:20:24.405958 24227 raft_consensus.cc:374] T 7479a6ee556a4056a0ce4d5a1ba4b323 P 3b728a70ee654948ab2fdf9f6d290630 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:20:24.405999 24227 raft_consensus.cc:729] T 7479a6ee556a4056a0ce4d5a1ba4b323 P 3b728a70ee654948ab2fdf9f6d290630 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b728a70ee654948ab2fdf9f6d290630, State: Initialized, Role: FOLLOWER
I20260809 16:20:24.406020 24231 ts_tablet_manager.cc:1428] T 7479a6ee556a4056a0ce4d5a1ba4b323 P 30b6ddb95a86440f92bac5652b288162: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260809 16:20:24.406075 24236 tablet_bootstrap.cc:492] T 7479a6ee556a4056a0ce4d5a1ba4b323 P c241eb2dc27b4e96a82c4b3feade2d7f: Bootstrap starting.
I20260809 16:20:24.406113 24227 consensus_queue.cc:260] T 7479a6ee556a4056a0ce4d5a1ba4b323 P 3b728a70ee654948ab2fdf9f6d290630 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c241eb2dc27b4e96a82c4b3feade2d7f" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 43627 } } peers { permanent_uuid: "3b728a70ee654948ab2fdf9f6d290630" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 41839 } } peers { permanent_uuid: "30b6ddb95a86440f92bac5652b288162" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 33979 } }
I20260809 16:20:24.406177 23758 heartbeater.cc:503] Master 127.21.51.190:33935 was elected leader, sending a full tablet report...
I20260809 16:20:24.406379 24227 ts_tablet_manager.cc:1428] T 7479a6ee556a4056a0ce4d5a1ba4b323 P 3b728a70ee654948ab2fdf9f6d290630: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 16:20:24.406451 23978 heartbeater.cc:503] Master 127.21.51.190:33935 was elected leader, sending a full tablet report...
I20260809 16:20:24.407194 24236 tablet_bootstrap.cc:654] T 7479a6ee556a4056a0ce4d5a1ba4b323 P c241eb2dc27b4e96a82c4b3feade2d7f: Neither blocks nor log segments found. Creating new log.
I20260809 16:20:24.407554 24236 log.cc:824] T 7479a6ee556a4056a0ce4d5a1ba4b323 P c241eb2dc27b4e96a82c4b3feade2d7f: Log is configured to *not* fsync() on all Append() calls
I20260809 16:20:24.408406 24236 tablet_bootstrap.cc:492] T 7479a6ee556a4056a0ce4d5a1ba4b323 P c241eb2dc27b4e96a82c4b3feade2d7f: No bootstrap required, opened a new log
I20260809 16:20:24.408496 24236 ts_tablet_manager.cc:1397] T 7479a6ee556a4056a0ce4d5a1ba4b323 P c241eb2dc27b4e96a82c4b3feade2d7f: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 16:20:24.409865 24236 raft_consensus.cc:348] T 7479a6ee556a4056a0ce4d5a1ba4b323 P c241eb2dc27b4e96a82c4b3feade2d7f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c241eb2dc27b4e96a82c4b3feade2d7f" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 43627 } } peers { permanent_uuid: "3b728a70ee654948ab2fdf9f6d290630" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 41839 } } peers { permanent_uuid: "30b6ddb95a86440f92bac5652b288162" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 33979 } }
I20260809 16:20:24.410003 24236 raft_consensus.cc:374] T 7479a6ee556a4056a0ce4d5a1ba4b323 P c241eb2dc27b4e96a82c4b3feade2d7f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:20:24.410048 24236 raft_consensus.cc:729] T 7479a6ee556a4056a0ce4d5a1ba4b323 P c241eb2dc27b4e96a82c4b3feade2d7f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c241eb2dc27b4e96a82c4b3feade2d7f, State: Initialized, Role: FOLLOWER
I20260809 16:20:24.410153 24236 consensus_queue.cc:260] T 7479a6ee556a4056a0ce4d5a1ba4b323 P c241eb2dc27b4e96a82c4b3feade2d7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c241eb2dc27b4e96a82c4b3feade2d7f" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 43627 } } peers { permanent_uuid: "3b728a70ee654948ab2fdf9f6d290630" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 41839 } } peers { permanent_uuid: "30b6ddb95a86440f92bac5652b288162" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 33979 } }
I20260809 16:20:24.410419 24236 ts_tablet_manager.cc:1428] T 7479a6ee556a4056a0ce4d5a1ba4b323 P c241eb2dc27b4e96a82c4b3feade2d7f: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260809 16:20:24.410493 24192 heartbeater.cc:503] Master 127.21.51.190:33935 was elected leader, sending a full tablet report...
W20260809 16:20:24.442284 23979 tablet.cc:2369] T 7479a6ee556a4056a0ce4d5a1ba4b323 P 3b728a70ee654948ab2fdf9f6d290630: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 16:20:24.532078 23761 tablet.cc:2369] T 7479a6ee556a4056a0ce4d5a1ba4b323 P 30b6ddb95a86440f92bac5652b288162: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 16:20:24.601373 24194 tablet.cc:2369] T 7479a6ee556a4056a0ce4d5a1ba4b323 P c241eb2dc27b4e96a82c4b3feade2d7f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 16:20:24.644034 24240 raft_consensus.cc:482] T 7479a6ee556a4056a0ce4d5a1ba4b323 P 3b728a70ee654948ab2fdf9f6d290630 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 16:20:24.644212 24240 raft_consensus.cc:504] T 7479a6ee556a4056a0ce4d5a1ba4b323 P 3b728a70ee654948ab2fdf9f6d290630 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c241eb2dc27b4e96a82c4b3feade2d7f" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 43627 } } peers { permanent_uuid: "3b728a70ee654948ab2fdf9f6d290630" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 41839 } } peers { permanent_uuid: "30b6ddb95a86440f92bac5652b288162" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 33979 } }
I20260809 16:20:24.644626 24240 leader_election.cc:288] T 7479a6ee556a4056a0ce4d5a1ba4b323 P 3b728a70ee654948ab2fdf9f6d290630 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c241eb2dc27b4e96a82c4b3feade2d7f (127.21.51.131:43627), 30b6ddb95a86440f92bac5652b288162 (127.21.51.129:33979)
I20260809 16:20:24.647866 24223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:24.645036 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.51.130:39619 (local address 127.21.51.131:43627)
0809 16:20:24.645086 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0809 16:20:24.645088 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:20:24.645208 (+ 120us) server_negotiation.cc:407] Connection header received
0809 16:20:24.645261 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:20:24.645262 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:20:24.645273 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:20:24.645339 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:20:24.645559 (+ 220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:24.646044 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:24.647111 (+ 1067us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:24.647471 (+ 360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:24.647530 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:24.647561 (+ 31us) server_negotiation.cc:1035] Waiting for connection context
0809 16:20:24.647745 (+ 184us) server_negotiation.cc:299] Negotiation successful
0809 16:20:24.647787 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260809 16:20:24.647898 23811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:24.644886 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.129:33979 (local address 127.21.51.130:43717)
0809 16:20:24.644968 (+ 82us) negotiation.cc:107] Waiting for socket to connect
0809 16:20:24.644980 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 16:20:24.645034 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:20:24.645225 (+ 191us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:20:24.645226 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:20:24.645228 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:20:24.645310 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:20:24.645312 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:20:24.646130 (+ 818us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:20:24.646132 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:20:24.646948 (+ 816us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:20:24.646951 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:20:24.647590 (+ 639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:20:24.647591 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:20:24.647676 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:24.647679 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 16:20:24.647753 (+ 74us) client_negotiation.cc:240] Negotiation successful
0809 16:20:24.647790 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43,"mutex_wait_us":42}
I20260809 16:20:24.647908 24220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:24.644979 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.51.130:43717 (local address 127.21.51.129:33979)
0809 16:20:24.645051 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0809 16:20:24.645053 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:20:24.645065 (+ 12us) server_negotiation.cc:407] Connection header received
0809 16:20:24.645100 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:20:24.645101 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:20:24.645112 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:20:24.645195 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:20:24.645403 (+ 208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:24.646037 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:24.647056 (+ 1019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:24.647478 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:24.647574 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:24.647583 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0809 16:20:24.647791 (+ 208us) server_negotiation.cc:299] Negotiation successful
0809 16:20:24.647840 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260809 16:20:24.647908 24243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:24.644920 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.131:43627 (local address 127.21.51.130:39619)
0809 16:20:24.645127 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0809 16:20:24.645138 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 16:20:24.645200 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:20:24.645385 (+ 185us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:20:24.645386 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:20:24.645389 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:20:24.645478 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:20:24.645480 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:20:24.646159 (+ 679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:20:24.646160 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:20:24.647015 (+ 855us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:20:24.647018 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:20:24.647565 (+ 547us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:20:24.647566 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:20:24.647664 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:24.647668 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 16:20:24.647715 (+ 47us) client_negotiation.cc:240] Negotiation successful
0809 16:20:24.647783 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":119,"threads_started":1}
I20260809 16:20:24.648294 23691 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7479a6ee556a4056a0ce4d5a1ba4b323" candidate_uuid: "3b728a70ee654948ab2fdf9f6d290630" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30b6ddb95a86440f92bac5652b288162" is_pre_election: true
I20260809 16:20:24.648425 24135 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7479a6ee556a4056a0ce4d5a1ba4b323" candidate_uuid: "3b728a70ee654948ab2fdf9f6d290630" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c241eb2dc27b4e96a82c4b3feade2d7f" is_pre_election: true
I20260809 16:20:24.648470 23691 raft_consensus.cc:2454] T 7479a6ee556a4056a0ce4d5a1ba4b323 P 30b6ddb95a86440f92bac5652b288162 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b728a70ee654948ab2fdf9f6d290630 in term 0.
I20260809 16:20:24.648581 24135 raft_consensus.cc:2454] T 7479a6ee556a4056a0ce4d5a1ba4b323 P c241eb2dc27b4e96a82c4b3feade2d7f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b728a70ee654948ab2fdf9f6d290630 in term 0.
I20260809 16:20:24.648784 23803 leader_election.cc:302] T 7479a6ee556a4056a0ce4d5a1ba4b323 P 3b728a70ee654948ab2fdf9f6d290630 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 30b6ddb95a86440f92bac5652b288162, 3b728a70ee654948ab2fdf9f6d290630; no voters:
I20260809 16:20:24.648983 24240 raft_consensus.cc:2781] T 7479a6ee556a4056a0ce4d5a1ba4b323 P 3b728a70ee654948ab2fdf9f6d290630 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 16:20:24.649056 24240 raft_consensus.cc:482] T 7479a6ee556a4056a0ce4d5a1ba4b323 P 3b728a70ee654948ab2fdf9f6d290630 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 16:20:24.649086 24240 raft_consensus.cc:3037] T 7479a6ee556a4056a0ce4d5a1ba4b323 P 3b728a70ee654948ab2fdf9f6d290630 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:20:24.650177 24240 raft_consensus.cc:504] T 7479a6ee556a4056a0ce4d5a1ba4b323 P 3b728a70ee654948ab2fdf9f6d290630 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c241eb2dc27b4e96a82c4b3feade2d7f" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 43627 } } peers { permanent_uuid: "3b728a70ee654948ab2fdf9f6d290630" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 41839 } } peers { permanent_uuid: "30b6ddb95a86440f92bac5652b288162" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 33979 } }
I20260809 16:20:24.650333 24240 leader_election.cc:288] T 7479a6ee556a4056a0ce4d5a1ba4b323 P 3b728a70ee654948ab2fdf9f6d290630 [CANDIDATE]: Term 1 election: Requested vote from peers c241eb2dc27b4e96a82c4b3feade2d7f (127.21.51.131:43627), 30b6ddb95a86440f92bac5652b288162 (127.21.51.129:33979)
I20260809 16:20:24.650542 24135 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7479a6ee556a4056a0ce4d5a1ba4b323" candidate_uuid: "3b728a70ee654948ab2fdf9f6d290630" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c241eb2dc27b4e96a82c4b3feade2d7f"
I20260809 16:20:24.650542 23691 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7479a6ee556a4056a0ce4d5a1ba4b323" candidate_uuid: "3b728a70ee654948ab2fdf9f6d290630" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30b6ddb95a86440f92bac5652b288162"
I20260809 16:20:24.650638 24135 raft_consensus.cc:3037] T 7479a6ee556a4056a0ce4d5a1ba4b323 P c241eb2dc27b4e96a82c4b3feade2d7f [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:20:24.650638 23691 raft_consensus.cc:3037] T 7479a6ee556a4056a0ce4d5a1ba4b323 P 30b6ddb95a86440f92bac5652b288162 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:20:24.651608 23691 raft_consensus.cc:2454] T 7479a6ee556a4056a0ce4d5a1ba4b323 P 30b6ddb95a86440f92bac5652b288162 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b728a70ee654948ab2fdf9f6d290630 in term 1.
I20260809 16:20:24.651608 24135 raft_consensus.cc:2454] T 7479a6ee556a4056a0ce4d5a1ba4b323 P c241eb2dc27b4e96a82c4b3feade2d7f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b728a70ee654948ab2fdf9f6d290630 in term 1.
I20260809 16:20:24.651804 23803 leader_election.cc:302] T 7479a6ee556a4056a0ce4d5a1ba4b323 P 3b728a70ee654948ab2fdf9f6d290630 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 30b6ddb95a86440f92bac5652b288162, 3b728a70ee654948ab2fdf9f6d290630; no voters:
I20260809 16:20:24.652060 24240 raft_consensus.cc:2781] T 7479a6ee556a4056a0ce4d5a1ba4b323 P 3b728a70ee654948ab2fdf9f6d290630 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:20:24.652218 24240 raft_consensus.cc:686] T 7479a6ee556a4056a0ce4d5a1ba4b323 P 3b728a70ee654948ab2fdf9f6d290630 [term 1 LEADER]: Becoming Leader. State: Replica: 3b728a70ee654948ab2fdf9f6d290630, State: Running, Role: LEADER
I20260809 16:20:24.652324 24240 consensus_queue.cc:237] T 7479a6ee556a4056a0ce4d5a1ba4b323 P 3b728a70ee654948ab2fdf9f6d290630 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c241eb2dc27b4e96a82c4b3feade2d7f" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 43627 } } peers { permanent_uuid: "3b728a70ee654948ab2fdf9f6d290630" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 41839 } } peers { permanent_uuid: "30b6ddb95a86440f92bac5652b288162" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 33979 } }
I20260809 16:20:24.653203 23436 catalog_manager.cc:5495] T 7479a6ee556a4056a0ce4d5a1ba4b323 P 3b728a70ee654948ab2fdf9f6d290630 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3b728a70ee654948ab2fdf9f6d290630 (127.21.51.130). New cstate: current_term: 1 leader_uuid: "3b728a70ee654948ab2fdf9f6d290630" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c241eb2dc27b4e96a82c4b3feade2d7f" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 43627 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3b728a70ee654948ab2fdf9f6d290630" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 41839 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "30b6ddb95a86440f92bac5652b288162" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 33979 } health_report { overall_health: UNKNOWN } } }
I20260809 16:20:24.679507 24226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:24.676944 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37718 (local address 127.21.51.130:41839)
0809 16:20:24.677005 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0809 16:20:24.677007 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:20:24.677015 (+ 8us) server_negotiation.cc:407] Connection header received
0809 16:20:24.677113 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:20:24.677114 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:20:24.677129 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:20:24.677188 (+ 59us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 16:20:24.677339 (+ 151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:24.677799 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:24.678658 (+ 859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:24.678837 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:24.678930 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:24.679136 (+ 206us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 16:20:24.679259 (+ 123us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 16:20:24.679322 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0809 16:20:24.679382 (+ 60us) server_negotiation.cc:299] Negotiation successful
0809 16:20:24.679400 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260809 16:20:24.681229 24135 raft_consensus.cc:1261] T 7479a6ee556a4056a0ce4d5a1ba4b323 P c241eb2dc27b4e96a82c4b3feade2d7f [term 1 FOLLOWER]: Refusing update from remote peer 3b728a70ee654948ab2fdf9f6d290630: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 16:20:24.681228 23691 raft_consensus.cc:1261] T 7479a6ee556a4056a0ce4d5a1ba4b323 P 30b6ddb95a86440f92bac5652b288162 [term 1 FOLLOWER]: Refusing update from remote peer 3b728a70ee654948ab2fdf9f6d290630: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 16:20:24.681591 24244 consensus_queue.cc:1035] T 7479a6ee556a4056a0ce4d5a1ba4b323 P 3b728a70ee654948ab2fdf9f6d290630 [LEADER]: Connected to new peer: Peer: permanent_uuid: "30b6ddb95a86440f92bac5652b288162" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 33979 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 16:20:24.681715 24240 consensus_queue.cc:1035] T 7479a6ee556a4056a0ce4d5a1ba4b323 P 3b728a70ee654948ab2fdf9f6d290630 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c241eb2dc27b4e96a82c4b3feade2d7f" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 43627 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 16:20:24.684115 24250 mvcc.cc:205] Tried to move back new op lower bound from 7316653771491631104 to 7316653771376492544. Current Snapshot: MvccSnapshot[applied={T|T < 7316653771491631104}]
I20260809 16:20:24.685814 24253 mvcc.cc:205] Tried to move back new op lower bound from 7316653771491631104 to 7316653771376492544. Current Snapshot: MvccSnapshot[applied={T|T < 7316653771491631104 or (T in {7316653771491631104})}]
I20260809 16:20:24.687600 23435 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55066:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 16:20:24.687690 23435 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55066:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 16:20:24.688443 23435 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 052d5e814a1d4882854b5c6f215a148b: Sending DeleteTablet for 3 replicas of tablet 7479a6ee556a4056a0ce4d5a1ba4b323
I20260809 16:20:24.688798 24099 tablet_service.cc:1514] Processing DeleteTablet for tablet 7479a6ee556a4056a0ce4d5a1ba4b323 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:20:24 UTC) from {username='kudu', principal='kudu/127.21.51.190@KRBTEST.COM'} at 127.0.0.1:47068
I20260809 16:20:24.688823 23669 tablet_service.cc:1514] Processing DeleteTablet for tablet 7479a6ee556a4056a0ce4d5a1ba4b323 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:20:24 UTC) from {username='kudu', principal='kudu/127.21.51.190@KRBTEST.COM'} at 127.0.0.1:48936
I20260809 16:20:24.689126 24271 tablet_replica.cc:330] T 7479a6ee556a4056a0ce4d5a1ba4b323 P 30b6ddb95a86440f92bac5652b288162: stopping tablet replica
I20260809 16:20:24.689119 23875 tablet_service.cc:1514] Processing DeleteTablet for tablet 7479a6ee556a4056a0ce4d5a1ba4b323 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:20:24 UTC) from {username='kudu', principal='kudu/127.21.51.190@KRBTEST.COM'} at 127.0.0.1:37706
I20260809 16:20:24.689066 23435 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55066:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 16:20:24.689209 23435 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 16:20:24.689415 24271 raft_consensus.cc:2229] T 7479a6ee556a4056a0ce4d5a1ba4b323 P 30b6ddb95a86440f92bac5652b288162 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 16:20:24.689460 24278 tablet_replica.cc:330] T 7479a6ee556a4056a0ce4d5a1ba4b323 P 3b728a70ee654948ab2fdf9f6d290630: stopping tablet replica
I20260809 16:20:24.689592 24271 raft_consensus.cc:2258] T 7479a6ee556a4056a0ce4d5a1ba4b323 P 30b6ddb95a86440f92bac5652b288162 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 16:20:24.689641 24278 raft_consensus.cc:2229] T 7479a6ee556a4056a0ce4d5a1ba4b323 P 3b728a70ee654948ab2fdf9f6d290630 [term 1 LEADER]: Raft consensus shutting down.
I20260809 16:20:24.689829 24278 raft_consensus.cc:2258] T 7479a6ee556a4056a0ce4d5a1ba4b323 P 3b728a70ee654948ab2fdf9f6d290630 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 16:20:24.690359 24278 ts_tablet_manager.cc:1905] T 7479a6ee556a4056a0ce4d5a1ba4b323 P 3b728a70ee654948ab2fdf9f6d290630: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 16:20:24.690652 23669 tablet_service.cc:1467] Processing CreateTablet for tablet 6367c459e7e1452abca2f541f4540cac (DEFAULT_TABLE table=test-table [id=66982997de8a429296f33083c2777d5f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:20:24.690765 23669 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6367c459e7e1452abca2f541f4540cac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:20:24.690994 24099 tablet_service.cc:1467] Processing CreateTablet for tablet 6367c459e7e1452abca2f541f4540cac (DEFAULT_TABLE table=test-table [id=66982997de8a429296f33083c2777d5f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:20:24.691120 24099 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6367c459e7e1452abca2f541f4540cac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:20:24.692113 24231 tablet_bootstrap.cc:492] T 6367c459e7e1452abca2f541f4540cac P 30b6ddb95a86440f92bac5652b288162: Bootstrap starting.
I20260809 16:20:24.692188 24278 ts_tablet_manager.cc:1918] T 7479a6ee556a4056a0ce4d5a1ba4b323 P 3b728a70ee654948ab2fdf9f6d290630: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 16:20:24.692252 24278 log.cc:1196] T 7479a6ee556a4056a0ce4d5a1ba4b323 P 3b728a70ee654948ab2fdf9f6d290630: Deleting WAL directory at /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786292417722232-21710-0/minicluster-data/ts-1/wal/wals/7479a6ee556a4056a0ce4d5a1ba4b323
I20260809 16:20:24.692487 24236 tablet_bootstrap.cc:492] T 6367c459e7e1452abca2f541f4540cac P c241eb2dc27b4e96a82c4b3feade2d7f: Bootstrap starting.
I20260809 16:20:24.692512 24231 tablet_bootstrap.cc:654] T 6367c459e7e1452abca2f541f4540cac P 30b6ddb95a86440f92bac5652b288162: Neither blocks nor log segments found. Creating new log.
I20260809 16:20:24.692544 24278 ts_tablet_manager.cc:1939] T 7479a6ee556a4056a0ce4d5a1ba4b323 P 3b728a70ee654948ab2fdf9f6d290630: Deleting consensus metadata
I20260809 16:20:24.692932 24236 tablet_bootstrap.cc:654] T 6367c459e7e1452abca2f541f4540cac P c241eb2dc27b4e96a82c4b3feade2d7f: Neither blocks nor log segments found. Creating new log.
I20260809 16:20:24.693444 23411 catalog_manager.cc:4841] TS 3b728a70ee654948ab2fdf9f6d290630 (127.21.51.130:41839): tablet 7479a6ee556a4056a0ce4d5a1ba4b323 (table test-table [id=95d260c896ff42159411523b140294d3]) successfully deleted
I20260809 16:20:24.693581 24271 ts_tablet_manager.cc:1905] T 7479a6ee556a4056a0ce4d5a1ba4b323 P 30b6ddb95a86440f92bac5652b288162: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 16:20:24.693696 23875 tablet_service.cc:1467] Processing CreateTablet for tablet 6367c459e7e1452abca2f541f4540cac (DEFAULT_TABLE table=test-table [id=66982997de8a429296f33083c2777d5f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:20:24.693809 23875 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6367c459e7e1452abca2f541f4540cac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:20:24.694872 24271 ts_tablet_manager.cc:1918] T 7479a6ee556a4056a0ce4d5a1ba4b323 P 30b6ddb95a86440f92bac5652b288162: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 16:20:24.694936 24271 log.cc:1196] T 7479a6ee556a4056a0ce4d5a1ba4b323 P 30b6ddb95a86440f92bac5652b288162: Deleting WAL directory at /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786292417722232-21710-0/minicluster-data/ts-0/wal/wals/7479a6ee556a4056a0ce4d5a1ba4b323
I20260809 16:20:24.695035 24231 tablet_bootstrap.cc:492] T 6367c459e7e1452abca2f541f4540cac P 30b6ddb95a86440f92bac5652b288162: No bootstrap required, opened a new log
I20260809 16:20:24.695072 24231 ts_tablet_manager.cc:1397] T 6367c459e7e1452abca2f541f4540cac P 30b6ddb95a86440f92bac5652b288162: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260809 16:20:24.695073 24227 tablet_bootstrap.cc:492] T 6367c459e7e1452abca2f541f4540cac P 3b728a70ee654948ab2fdf9f6d290630: Bootstrap starting.
I20260809 16:20:24.695227 24231 raft_consensus.cc:348] T 6367c459e7e1452abca2f541f4540cac P 30b6ddb95a86440f92bac5652b288162 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30b6ddb95a86440f92bac5652b288162" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 33979 } } peers { permanent_uuid: "3b728a70ee654948ab2fdf9f6d290630" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 41839 } } peers { permanent_uuid: "c241eb2dc27b4e96a82c4b3feade2d7f" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 43627 } }
I20260809 16:20:24.695312 24231 raft_consensus.cc:374] T 6367c459e7e1452abca2f541f4540cac P 30b6ddb95a86440f92bac5652b288162 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:20:24.695331 24231 raft_consensus.cc:729] T 6367c459e7e1452abca2f541f4540cac P 30b6ddb95a86440f92bac5652b288162 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 30b6ddb95a86440f92bac5652b288162, State: Initialized, Role: FOLLOWER
I20260809 16:20:24.695402 24231 consensus_queue.cc:260] T 6367c459e7e1452abca2f541f4540cac P 30b6ddb95a86440f92bac5652b288162 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30b6ddb95a86440f92bac5652b288162" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 33979 } } peers { permanent_uuid: "3b728a70ee654948ab2fdf9f6d290630" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 41839 } } peers { permanent_uuid: "c241eb2dc27b4e96a82c4b3feade2d7f" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 43627 } }
I20260809 16:20:24.695472 24227 tablet_bootstrap.cc:654] T 6367c459e7e1452abca2f541f4540cac P 3b728a70ee654948ab2fdf9f6d290630: Neither blocks nor log segments found. Creating new log.
I20260809 16:20:24.695513 24231 ts_tablet_manager.cc:1428] T 6367c459e7e1452abca2f541f4540cac P 30b6ddb95a86440f92bac5652b288162: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 16:20:24.695219 24271 ts_tablet_manager.cc:1939] T 7479a6ee556a4056a0ce4d5a1ba4b323 P 30b6ddb95a86440f92bac5652b288162: Deleting consensus metadata
I20260809 16:20:24.695945 23413 catalog_manager.cc:4841] TS 30b6ddb95a86440f92bac5652b288162 (127.21.51.129:33979): tablet 7479a6ee556a4056a0ce4d5a1ba4b323 (table test-table [id=95d260c896ff42159411523b140294d3]) successfully deleted
I20260809 16:20:24.696482 24227 tablet_bootstrap.cc:492] T 6367c459e7e1452abca2f541f4540cac P 3b728a70ee654948ab2fdf9f6d290630: No bootstrap required, opened a new log
I20260809 16:20:24.696535 24227 ts_tablet_manager.cc:1397] T 6367c459e7e1452abca2f541f4540cac P 3b728a70ee654948ab2fdf9f6d290630: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260809 16:20:24.696666 24227 raft_consensus.cc:348] T 6367c459e7e1452abca2f541f4540cac P 3b728a70ee654948ab2fdf9f6d290630 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30b6ddb95a86440f92bac5652b288162" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 33979 } } peers { permanent_uuid: "3b728a70ee654948ab2fdf9f6d290630" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 41839 } } peers { permanent_uuid: "c241eb2dc27b4e96a82c4b3feade2d7f" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 43627 } }
I20260809 16:20:24.696755 24227 raft_consensus.cc:374] T 6367c459e7e1452abca2f541f4540cac P 3b728a70ee654948ab2fdf9f6d290630 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:20:24.696776 24227 raft_consensus.cc:729] T 6367c459e7e1452abca2f541f4540cac P 3b728a70ee654948ab2fdf9f6d290630 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b728a70ee654948ab2fdf9f6d290630, State: Initialized, Role: FOLLOWER
I20260809 16:20:24.696784 24274 tablet_replica.cc:330] T 7479a6ee556a4056a0ce4d5a1ba4b323 P c241eb2dc27b4e96a82c4b3feade2d7f: stopping tablet replica
I20260809 16:20:24.696821 24227 consensus_queue.cc:260] T 6367c459e7e1452abca2f541f4540cac P 3b728a70ee654948ab2fdf9f6d290630 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30b6ddb95a86440f92bac5652b288162" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 33979 } } peers { permanent_uuid: "3b728a70ee654948ab2fdf9f6d290630" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 41839 } } peers { permanent_uuid: "c241eb2dc27b4e96a82c4b3feade2d7f" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 43627 } }
I20260809 16:20:24.696905 24227 ts_tablet_manager.cc:1428] T 6367c459e7e1452abca2f541f4540cac P 3b728a70ee654948ab2fdf9f6d290630: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 16:20:24.696899 24274 raft_consensus.cc:2229] T 7479a6ee556a4056a0ce4d5a1ba4b323 P c241eb2dc27b4e96a82c4b3feade2d7f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 16:20:24.697117 24274 raft_consensus.cc:2258] T 7479a6ee556a4056a0ce4d5a1ba4b323 P c241eb2dc27b4e96a82c4b3feade2d7f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 16:20:24.697398 24236 tablet_bootstrap.cc:492] T 6367c459e7e1452abca2f541f4540cac P c241eb2dc27b4e96a82c4b3feade2d7f: No bootstrap required, opened a new log
I20260809 16:20:24.697454 24236 ts_tablet_manager.cc:1397] T 6367c459e7e1452abca2f541f4540cac P c241eb2dc27b4e96a82c4b3feade2d7f: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.001s
I20260809 16:20:24.697598 24236 raft_consensus.cc:348] T 6367c459e7e1452abca2f541f4540cac P c241eb2dc27b4e96a82c4b3feade2d7f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30b6ddb95a86440f92bac5652b288162" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 33979 } } peers { permanent_uuid: "3b728a70ee654948ab2fdf9f6d290630" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 41839 } } peers { permanent_uuid: "c241eb2dc27b4e96a82c4b3feade2d7f" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 43627 } }
I20260809 16:20:24.697661 24236 raft_consensus.cc:374] T 6367c459e7e1452abca2f541f4540cac P c241eb2dc27b4e96a82c4b3feade2d7f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:20:24.697677 24274 ts_tablet_manager.cc:1905] T 7479a6ee556a4056a0ce4d5a1ba4b323 P c241eb2dc27b4e96a82c4b3feade2d7f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 16:20:24.697683 24236 raft_consensus.cc:729] T 6367c459e7e1452abca2f541f4540cac P c241eb2dc27b4e96a82c4b3feade2d7f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c241eb2dc27b4e96a82c4b3feade2d7f, State: Initialized, Role: FOLLOWER
I20260809 16:20:24.697872 24236 consensus_queue.cc:260] T 6367c459e7e1452abca2f541f4540cac P c241eb2dc27b4e96a82c4b3feade2d7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30b6ddb95a86440f92bac5652b288162" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 33979 } } peers { permanent_uuid: "3b728a70ee654948ab2fdf9f6d290630" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 41839 } } peers { permanent_uuid: "c241eb2dc27b4e96a82c4b3feade2d7f" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 43627 } }
I20260809 16:20:24.697974 24236 ts_tablet_manager.cc:1428] T 6367c459e7e1452abca2f541f4540cac P c241eb2dc27b4e96a82c4b3feade2d7f: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 16:20:24.699123 24274 ts_tablet_manager.cc:1918] T 7479a6ee556a4056a0ce4d5a1ba4b323 P c241eb2dc27b4e96a82c4b3feade2d7f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 16:20:24.699220 24274 log.cc:1196] T 7479a6ee556a4056a0ce4d5a1ba4b323 P c241eb2dc27b4e96a82c4b3feade2d7f: Deleting WAL directory at /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786292417722232-21710-0/minicluster-data/ts-2/wal/wals/7479a6ee556a4056a0ce4d5a1ba4b323
I20260809 16:20:24.699469 24274 ts_tablet_manager.cc:1939] T 7479a6ee556a4056a0ce4d5a1ba4b323 P c241eb2dc27b4e96a82c4b3feade2d7f: Deleting consensus metadata
I20260809 16:20:24.699899 23414 catalog_manager.cc:4841] TS c241eb2dc27b4e96a82c4b3feade2d7f (127.21.51.131:43627): tablet 7479a6ee556a4056a0ce4d5a1ba4b323 (table test-table [id=95d260c896ff42159411523b140294d3]) successfully deleted
I20260809 16:20:24.888298 24242 raft_consensus.cc:482] T 6367c459e7e1452abca2f541f4540cac P c241eb2dc27b4e96a82c4b3feade2d7f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 16:20:24.888402 24242 raft_consensus.cc:504] T 6367c459e7e1452abca2f541f4540cac P c241eb2dc27b4e96a82c4b3feade2d7f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30b6ddb95a86440f92bac5652b288162" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 33979 } } peers { permanent_uuid: "3b728a70ee654948ab2fdf9f6d290630" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 41839 } } peers { permanent_uuid: "c241eb2dc27b4e96a82c4b3feade2d7f" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 43627 } }
I20260809 16:20:24.888808 24242 leader_election.cc:288] T 6367c459e7e1452abca2f541f4540cac P c241eb2dc27b4e96a82c4b3feade2d7f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 30b6ddb95a86440f92bac5652b288162 (127.21.51.129:33979), 3b728a70ee654948ab2fdf9f6d290630 (127.21.51.130:41839)
I20260809 16:20:24.892846 24226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:24.889134 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.51.131:43417 (local address 127.21.51.130:41839)
0809 16:20:24.889184 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0809 16:20:24.889194 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 16:20:24.889203 (+ 9us) server_negotiation.cc:407] Connection header received
0809 16:20:24.889253 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:20:24.889254 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:20:24.889266 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:20:24.889321 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:20:24.889570 (+ 249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:24.890344 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:24.891901 (+ 1557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:24.892457 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:24.892534 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:24.892543 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0809 16:20:24.892733 (+ 190us) server_negotiation.cc:299] Negotiation successful
0809 16:20:24.892773 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260809 16:20:24.892846 24220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:24.889057 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.51.131:60909 (local address 127.21.51.129:33979)
0809 16:20:24.889115 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0809 16:20:24.889118 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 16:20:24.889179 (+ 61us) server_negotiation.cc:407] Connection header received
0809 16:20:24.889254 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:20:24.889254 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:20:24.889265 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:20:24.889321 (+ 56us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:20:24.889570 (+ 249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:24.890335 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:24.891903 (+ 1568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:24.892457 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:24.892534 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:24.892543 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0809 16:20:24.892739 (+ 196us) server_negotiation.cc:299] Negotiation successful
0809 16:20:24.892775 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260809 16:20:24.892887 24289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:24.888995 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.130:41839 (local address 127.21.51.131:43417)
0809 16:20:24.889117 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0809 16:20:24.889125 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0809 16:20:24.889179 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:20:24.889349 (+ 170us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:20:24.889349 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:20:24.889352 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:20:24.889457 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:20:24.889458 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:20:24.890431 (+ 973us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:20:24.890432 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:20:24.891806 (+ 1374us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:20:24.891809 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:20:24.892553 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:20:24.892554 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:20:24.892631 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:24.892638 (+ 7us) client_negotiation.cc:769] Sending connection context
0809 16:20:24.892715 (+ 77us) client_negotiation.cc:240] Negotiation successful
0809 16:20:24.892777 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111,"mutex_wait_us":81,"thread_start_us":58,"threads_started":1}
I20260809 16:20:24.892906 24290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:24.888981 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.129:33979 (local address 127.21.51.131:60909)
0809 16:20:24.889103 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0809 16:20:24.889116 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 16:20:24.889179 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:20:24.889348 (+ 169us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:20:24.889349 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:20:24.889352 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:20:24.889456 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:20:24.889458 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:20:24.890432 (+ 974us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:20:24.890432 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:20:24.891806 (+ 1374us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:20:24.891809 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:20:24.892552 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:20:24.892554 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:20:24.892631 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:24.892638 (+ 7us) client_negotiation.cc:769] Sending connection context
0809 16:20:24.892715 (+ 77us) client_negotiation.cc:240] Negotiation successful
0809 16:20:24.892777 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":100,"mutex_wait_us":91,"thread_start_us":59,"threads_started":1}
I20260809 16:20:24.893352 23691 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6367c459e7e1452abca2f541f4540cac" candidate_uuid: "c241eb2dc27b4e96a82c4b3feade2d7f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30b6ddb95a86440f92bac5652b288162" is_pre_election: true
I20260809 16:20:24.893438 23691 raft_consensus.cc:2454] T 6367c459e7e1452abca2f541f4540cac P 30b6ddb95a86440f92bac5652b288162 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c241eb2dc27b4e96a82c4b3feade2d7f in term 0.
I20260809 16:20:24.893348 23912 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6367c459e7e1452abca2f541f4540cac" candidate_uuid: "c241eb2dc27b4e96a82c4b3feade2d7f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b728a70ee654948ab2fdf9f6d290630" is_pre_election: true
I20260809 16:20:24.893494 23912 raft_consensus.cc:2454] T 6367c459e7e1452abca2f541f4540cac P 3b728a70ee654948ab2fdf9f6d290630 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c241eb2dc27b4e96a82c4b3feade2d7f in term 0.
I20260809 16:20:24.893635 24024 leader_election.cc:302] T 6367c459e7e1452abca2f541f4540cac P c241eb2dc27b4e96a82c4b3feade2d7f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 30b6ddb95a86440f92bac5652b288162, c241eb2dc27b4e96a82c4b3feade2d7f; no voters:
I20260809 16:20:24.893807 24242 raft_consensus.cc:2781] T 6367c459e7e1452abca2f541f4540cac P c241eb2dc27b4e96a82c4b3feade2d7f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 16:20:24.893854 24242 raft_consensus.cc:482] T 6367c459e7e1452abca2f541f4540cac P c241eb2dc27b4e96a82c4b3feade2d7f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 16:20:24.893878 24242 raft_consensus.cc:3037] T 6367c459e7e1452abca2f541f4540cac P c241eb2dc27b4e96a82c4b3feade2d7f [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:20:24.894645 24242 raft_consensus.cc:504] T 6367c459e7e1452abca2f541f4540cac P c241eb2dc27b4e96a82c4b3feade2d7f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30b6ddb95a86440f92bac5652b288162" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 33979 } } peers { permanent_uuid: "3b728a70ee654948ab2fdf9f6d290630" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 41839 } } peers { permanent_uuid: "c241eb2dc27b4e96a82c4b3feade2d7f" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 43627 } }
I20260809 16:20:24.894790 24242 leader_election.cc:288] T 6367c459e7e1452abca2f541f4540cac P c241eb2dc27b4e96a82c4b3feade2d7f [CANDIDATE]: Term 1 election: Requested vote from peers 30b6ddb95a86440f92bac5652b288162 (127.21.51.129:33979), 3b728a70ee654948ab2fdf9f6d290630 (127.21.51.130:41839)
I20260809 16:20:24.894937 23691 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6367c459e7e1452abca2f541f4540cac" candidate_uuid: "c241eb2dc27b4e96a82c4b3feade2d7f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30b6ddb95a86440f92bac5652b288162"
I20260809 16:20:24.894954 23912 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6367c459e7e1452abca2f541f4540cac" candidate_uuid: "c241eb2dc27b4e96a82c4b3feade2d7f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b728a70ee654948ab2fdf9f6d290630"
I20260809 16:20:24.895011 23691 raft_consensus.cc:3037] T 6367c459e7e1452abca2f541f4540cac P 30b6ddb95a86440f92bac5652b288162 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:20:24.895033 23912 raft_consensus.cc:3037] T 6367c459e7e1452abca2f541f4540cac P 3b728a70ee654948ab2fdf9f6d290630 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:20:24.895733 23912 raft_consensus.cc:2454] T 6367c459e7e1452abca2f541f4540cac P 3b728a70ee654948ab2fdf9f6d290630 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c241eb2dc27b4e96a82c4b3feade2d7f in term 1.
I20260809 16:20:24.895920 24021 leader_election.cc:302] T 6367c459e7e1452abca2f541f4540cac P c241eb2dc27b4e96a82c4b3feade2d7f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b728a70ee654948ab2fdf9f6d290630, c241eb2dc27b4e96a82c4b3feade2d7f; no voters:
I20260809 16:20:24.896034 24242 raft_consensus.cc:2781] T 6367c459e7e1452abca2f541f4540cac P c241eb2dc27b4e96a82c4b3feade2d7f [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:20:24.896047 23691 raft_consensus.cc:2454] T 6367c459e7e1452abca2f541f4540cac P 30b6ddb95a86440f92bac5652b288162 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c241eb2dc27b4e96a82c4b3feade2d7f in term 1.
I20260809 16:20:24.896167 24242 raft_consensus.cc:686] T 6367c459e7e1452abca2f541f4540cac P c241eb2dc27b4e96a82c4b3feade2d7f [term 1 LEADER]: Becoming Leader. State: Replica: c241eb2dc27b4e96a82c4b3feade2d7f, State: Running, Role: LEADER
I20260809 16:20:24.896242 24242 consensus_queue.cc:237] T 6367c459e7e1452abca2f541f4540cac P c241eb2dc27b4e96a82c4b3feade2d7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30b6ddb95a86440f92bac5652b288162" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 33979 } } peers { permanent_uuid: "3b728a70ee654948ab2fdf9f6d290630" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 41839 } } peers { permanent_uuid: "c241eb2dc27b4e96a82c4b3feade2d7f" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 43627 } }
I20260809 16:20:24.896994 23435 catalog_manager.cc:5495] T 6367c459e7e1452abca2f541f4540cac P c241eb2dc27b4e96a82c4b3feade2d7f reported cstate change: term changed from 0 to 1, leader changed from <none> to c241eb2dc27b4e96a82c4b3feade2d7f (127.21.51.131). New cstate: current_term: 1 leader_uuid: "c241eb2dc27b4e96a82c4b3feade2d7f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30b6ddb95a86440f92bac5652b288162" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 33979 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3b728a70ee654948ab2fdf9f6d290630" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 41839 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c241eb2dc27b4e96a82c4b3feade2d7f" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 43627 } health_report { overall_health: HEALTHY } } }
Aug 09 16:20:24 dist-test-slave-s11t krb5kdc[23361](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292423, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.51.190@KRBTEST.COM for kudu/127.21.51.190@KRBTEST.COM
I20260809 16:20:24.948845 24306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:24.941563 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45502 (local address 127.21.51.190:33935)
0809 16:20:24.941809 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0809 16:20:24.941812 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 16:20:24.941915 (+ 103us) server_negotiation.cc:407] Connection header received
0809 16:20:24.942008 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:20:24.942009 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:20:24.942020 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:20:24.942110 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:20:24.942609 (+ 499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:24.943123 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:24.943848 (+ 725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:24.944038 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:24.944114 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:24.946261 (+ 2147us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:20:24.946262 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:20:24.946263 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:20:24.946270 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:20:24.947531 (+ 1261us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:20:24.947948 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:20:24.947950 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:20:24.947950 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:20:24.948016 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:20:24.948214 (+ 198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:20:24.948214 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:20:24.948215 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:20:24.948324 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:20:24.948402 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0809 16:20:24.948566 (+ 164us) server_negotiation.cc:299] Negotiation successful
0809 16:20:24.948726 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":2,"server-negotiator.queue_time_us":215,"thread_start_us":70,"threads_started":1}
I20260809 16:20:24.948905 24304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:24.941453 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:33935 (local address 127.0.0.1:45502)
0809 16:20:24.941823 (+ 370us) negotiation.cc:107] Waiting for socket to connect
0809 16:20:24.941836 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 16:20:24.941931 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:20:24.942115 (+ 184us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:20:24.942116 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:20:24.942459 (+ 343us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:20:24.942506 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:20:24.942508 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:20:24.943225 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:20:24.943226 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:20:24.943756 (+ 530us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:20:24.943758 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:20:24.944124 (+ 366us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:20:24.944125 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:20:24.944187 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:24.944770 (+ 583us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:20:24.944775 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:20:24.946166 (+ 1391us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:20:24.947630 (+ 1464us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:20:24.947638 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:20:24.947641 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:20:24.947869 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:20:24.948085 (+ 216us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:20:24.948085 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:20:24.948086 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:20:24.948148 (+ 62us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:20:24.948402 (+ 254us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:20:24.948410 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:20:24.948457 (+ 47us) client_negotiation.cc:769] Sending connection context
0809 16:20:24.948542 (+ 85us) client_negotiation.cc:240] Negotiation successful
0809 16:20:24.948785 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":172,"mutex_wait_us":137,"thread_start_us":137,"threads_started":1}
I20260809 16:20:24.951072 23435 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.21.51.190@KRBTEST.COM'} at 127.0.0.1:45502:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260809 16:20:24.951298 23435 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 16:20:24.952873 23875 tablet_service.cc:1467] Processing CreateTablet for tablet 461a274e25f64897aab71acc64e0b9ff (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=26b33641792e4fb6a40cd92f7ccb6352]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 16:20:24.952898 24099 tablet_service.cc:1467] Processing CreateTablet for tablet 461a274e25f64897aab71acc64e0b9ff (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=26b33641792e4fb6a40cd92f7ccb6352]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 16:20:24.952940 23669 tablet_service.cc:1467] Processing CreateTablet for tablet 461a274e25f64897aab71acc64e0b9ff (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=26b33641792e4fb6a40cd92f7ccb6352]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 16:20:24.953042 24099 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 461a274e25f64897aab71acc64e0b9ff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:20:24.953044 23875 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 461a274e25f64897aab71acc64e0b9ff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:20:24.953118 23669 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 461a274e25f64897aab71acc64e0b9ff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:20:24.954581 24227 tablet_bootstrap.cc:492] T 461a274e25f64897aab71acc64e0b9ff P 3b728a70ee654948ab2fdf9f6d290630: Bootstrap starting.
I20260809 16:20:24.954622 24231 tablet_bootstrap.cc:492] T 461a274e25f64897aab71acc64e0b9ff P 30b6ddb95a86440f92bac5652b288162: Bootstrap starting.
I20260809 16:20:24.955030 24227 tablet_bootstrap.cc:654] T 461a274e25f64897aab71acc64e0b9ff P 3b728a70ee654948ab2fdf9f6d290630: Neither blocks nor log segments found. Creating new log.
I20260809 16:20:24.955067 24231 tablet_bootstrap.cc:654] T 461a274e25f64897aab71acc64e0b9ff P 30b6ddb95a86440f92bac5652b288162: Neither blocks nor log segments found. Creating new log.
I20260809 16:20:24.955314 24236 tablet_bootstrap.cc:492] T 461a274e25f64897aab71acc64e0b9ff P c241eb2dc27b4e96a82c4b3feade2d7f: Bootstrap starting.
I20260809 16:20:24.955636 24227 tablet_bootstrap.cc:492] T 461a274e25f64897aab71acc64e0b9ff P 3b728a70ee654948ab2fdf9f6d290630: No bootstrap required, opened a new log
I20260809 16:20:24.955691 24227 ts_tablet_manager.cc:1397] T 461a274e25f64897aab71acc64e0b9ff P 3b728a70ee654948ab2fdf9f6d290630: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260809 16:20:24.955780 24236 tablet_bootstrap.cc:654] T 461a274e25f64897aab71acc64e0b9ff P c241eb2dc27b4e96a82c4b3feade2d7f: Neither blocks nor log segments found. Creating new log.
I20260809 16:20:24.955790 24231 tablet_bootstrap.cc:492] T 461a274e25f64897aab71acc64e0b9ff P 30b6ddb95a86440f92bac5652b288162: No bootstrap required, opened a new log
I20260809 16:20:24.955830 24231 ts_tablet_manager.cc:1397] T 461a274e25f64897aab71acc64e0b9ff P 30b6ddb95a86440f92bac5652b288162: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260809 16:20:24.955824 24227 raft_consensus.cc:348] T 461a274e25f64897aab71acc64e0b9ff P 3b728a70ee654948ab2fdf9f6d290630 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30b6ddb95a86440f92bac5652b288162" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 33979 } } peers { permanent_uuid: "3b728a70ee654948ab2fdf9f6d290630" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 41839 } } peers { permanent_uuid: "c241eb2dc27b4e96a82c4b3feade2d7f" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 43627 } }
I20260809 16:20:24.955900 24227 raft_consensus.cc:374] T 461a274e25f64897aab71acc64e0b9ff P 3b728a70ee654948ab2fdf9f6d290630 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:20:24.955926 24227 raft_consensus.cc:729] T 461a274e25f64897aab71acc64e0b9ff P 3b728a70ee654948ab2fdf9f6d290630 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b728a70ee654948ab2fdf9f6d290630, State: Initialized, Role: FOLLOWER
I20260809 16:20:24.955966 24227 consensus_queue.cc:260] T 461a274e25f64897aab71acc64e0b9ff P 3b728a70ee654948ab2fdf9f6d290630 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30b6ddb95a86440f92bac5652b288162" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 33979 } } peers { permanent_uuid: "3b728a70ee654948ab2fdf9f6d290630" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 41839 } } peers { permanent_uuid: "c241eb2dc27b4e96a82c4b3feade2d7f" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 43627 } }
I20260809 16:20:24.955974 24231 raft_consensus.cc:348] T 461a274e25f64897aab71acc64e0b9ff P 30b6ddb95a86440f92bac5652b288162 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30b6ddb95a86440f92bac5652b288162" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 33979 } } peers { permanent_uuid: "3b728a70ee654948ab2fdf9f6d290630" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 41839 } } peers { permanent_uuid: "c241eb2dc27b4e96a82c4b3feade2d7f" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 43627 } }
I20260809 16:20:24.956036 24231 raft_consensus.cc:374] T 461a274e25f64897aab71acc64e0b9ff P 30b6ddb95a86440f92bac5652b288162 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:20:24.956058 24231 raft_consensus.cc:729] T 461a274e25f64897aab71acc64e0b9ff P 30b6ddb95a86440f92bac5652b288162 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 30b6ddb95a86440f92bac5652b288162, State: Initialized, Role: FOLLOWER
I20260809 16:20:24.956065 24227 ts_tablet_manager.cc:1428] T 461a274e25f64897aab71acc64e0b9ff P 3b728a70ee654948ab2fdf9f6d290630: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 16:20:24.956100 24231 consensus_queue.cc:260] T 461a274e25f64897aab71acc64e0b9ff P 30b6ddb95a86440f92bac5652b288162 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30b6ddb95a86440f92bac5652b288162" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 33979 } } peers { permanent_uuid: "3b728a70ee654948ab2fdf9f6d290630" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 41839 } } peers { permanent_uuid: "c241eb2dc27b4e96a82c4b3feade2d7f" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 43627 } }
I20260809 16:20:24.956117 24261 tablet_replica.cc:439] T 461a274e25f64897aab71acc64e0b9ff P 3b728a70ee654948ab2fdf9f6d290630: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30b6ddb95a86440f92bac5652b288162" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 33979 } } peers { permanent_uuid: "3b728a70ee654948ab2fdf9f6d290630" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 41839 } } peers { permanent_uuid: "c241eb2dc27b4e96a82c4b3feade2d7f" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 43627 } } }
I20260809 16:20:24.956195 24231 ts_tablet_manager.cc:1428] T 461a274e25f64897aab71acc64e0b9ff P 30b6ddb95a86440f92bac5652b288162: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 16:20:24.956229 24261 tablet_replica.cc:442] T 461a274e25f64897aab71acc64e0b9ff P 3b728a70ee654948ab2fdf9f6d290630: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 16:20:24.956382 24238 tablet_replica.cc:439] T 461a274e25f64897aab71acc64e0b9ff P 30b6ddb95a86440f92bac5652b288162: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30b6ddb95a86440f92bac5652b288162" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 33979 } } peers { permanent_uuid: "3b728a70ee654948ab2fdf9f6d290630" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 41839 } } peers { permanent_uuid: "c241eb2dc27b4e96a82c4b3feade2d7f" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 43627 } } }
I20260809 16:20:24.956471 24238 tablet_replica.cc:442] T 461a274e25f64897aab71acc64e0b9ff P 30b6ddb95a86440f92bac5652b288162: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 16:20:24.956709 24236 tablet_bootstrap.cc:492] T 461a274e25f64897aab71acc64e0b9ff P c241eb2dc27b4e96a82c4b3feade2d7f: No bootstrap required, opened a new log
I20260809 16:20:24.956770 24236 ts_tablet_manager.cc:1397] T 461a274e25f64897aab71acc64e0b9ff P c241eb2dc27b4e96a82c4b3feade2d7f: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260809 16:20:24.956930 24236 raft_consensus.cc:348] T 461a274e25f64897aab71acc64e0b9ff P c241eb2dc27b4e96a82c4b3feade2d7f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30b6ddb95a86440f92bac5652b288162" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 33979 } } peers { permanent_uuid: "3b728a70ee654948ab2fdf9f6d290630" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 41839 } } peers { permanent_uuid: "c241eb2dc27b4e96a82c4b3feade2d7f" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 43627 } }
I20260809 16:20:24.957013 24236 raft_consensus.cc:374] T 461a274e25f64897aab71acc64e0b9ff P c241eb2dc27b4e96a82c4b3feade2d7f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:20:24.957036 24236 raft_consensus.cc:729] T 461a274e25f64897aab71acc64e0b9ff P c241eb2dc27b4e96a82c4b3feade2d7f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c241eb2dc27b4e96a82c4b3feade2d7f, State: Initialized, Role: FOLLOWER
I20260809 16:20:24.957085 24236 consensus_queue.cc:260] T 461a274e25f64897aab71acc64e0b9ff P c241eb2dc27b4e96a82c4b3feade2d7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30b6ddb95a86440f92bac5652b288162" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 33979 } } peers { permanent_uuid: "3b728a70ee654948ab2fdf9f6d290630" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 41839 } } peers { permanent_uuid: "c241eb2dc27b4e96a82c4b3feade2d7f" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 43627 } }
I20260809 16:20:24.957181 24236 ts_tablet_manager.cc:1428] T 461a274e25f64897aab71acc64e0b9ff P c241eb2dc27b4e96a82c4b3feade2d7f: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 16:20:24.957219 24242 tablet_replica.cc:439] T 461a274e25f64897aab71acc64e0b9ff P c241eb2dc27b4e96a82c4b3feade2d7f: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30b6ddb95a86440f92bac5652b288162" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 33979 } } peers { permanent_uuid: "3b728a70ee654948ab2fdf9f6d290630" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 41839 } } peers { permanent_uuid: "c241eb2dc27b4e96a82c4b3feade2d7f" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 43627 } } }
I20260809 16:20:24.957314 24242 tablet_replica.cc:442] T 461a274e25f64897aab71acc64e0b9ff P c241eb2dc27b4e96a82c4b3feade2d7f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 16:20:25.104600 24242 raft_consensus.cc:482] T 461a274e25f64897aab71acc64e0b9ff P c241eb2dc27b4e96a82c4b3feade2d7f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 16:20:25.104719 24242 raft_consensus.cc:504] T 461a274e25f64897aab71acc64e0b9ff P c241eb2dc27b4e96a82c4b3feade2d7f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30b6ddb95a86440f92bac5652b288162" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 33979 } } peers { permanent_uuid: "3b728a70ee654948ab2fdf9f6d290630" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 41839 } } peers { permanent_uuid: "c241eb2dc27b4e96a82c4b3feade2d7f" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 43627 } }
I20260809 16:20:25.104889 24242 leader_election.cc:288] T 461a274e25f64897aab71acc64e0b9ff P c241eb2dc27b4e96a82c4b3feade2d7f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 30b6ddb95a86440f92bac5652b288162 (127.21.51.129:33979), 3b728a70ee654948ab2fdf9f6d290630 (127.21.51.130:41839)
I20260809 16:20:25.105110 23912 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "461a274e25f64897aab71acc64e0b9ff" candidate_uuid: "c241eb2dc27b4e96a82c4b3feade2d7f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b728a70ee654948ab2fdf9f6d290630" is_pre_election: true
I20260809 16:20:25.105149 23691 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "461a274e25f64897aab71acc64e0b9ff" candidate_uuid: "c241eb2dc27b4e96a82c4b3feade2d7f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30b6ddb95a86440f92bac5652b288162" is_pre_election: true
I20260809 16:20:25.105223 23912 raft_consensus.cc:2454] T 461a274e25f64897aab71acc64e0b9ff P 3b728a70ee654948ab2fdf9f6d290630 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c241eb2dc27b4e96a82c4b3feade2d7f in term 0.
I20260809 16:20:25.105249 23691 raft_consensus.cc:2454] T 461a274e25f64897aab71acc64e0b9ff P 30b6ddb95a86440f92bac5652b288162 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c241eb2dc27b4e96a82c4b3feade2d7f in term 0.
I20260809 16:20:25.105407 24021 leader_election.cc:302] T 461a274e25f64897aab71acc64e0b9ff P c241eb2dc27b4e96a82c4b3feade2d7f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b728a70ee654948ab2fdf9f6d290630, c241eb2dc27b4e96a82c4b3feade2d7f; no voters:
I20260809 16:20:25.105527 24242 raft_consensus.cc:2781] T 461a274e25f64897aab71acc64e0b9ff P c241eb2dc27b4e96a82c4b3feade2d7f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 16:20:25.105571 24242 raft_consensus.cc:482] T 461a274e25f64897aab71acc64e0b9ff P c241eb2dc27b4e96a82c4b3feade2d7f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 16:20:25.105587 24242 raft_consensus.cc:3037] T 461a274e25f64897aab71acc64e0b9ff P c241eb2dc27b4e96a82c4b3feade2d7f [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:20:25.106338 24242 raft_consensus.cc:504] T 461a274e25f64897aab71acc64e0b9ff P c241eb2dc27b4e96a82c4b3feade2d7f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30b6ddb95a86440f92bac5652b288162" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 33979 } } peers { permanent_uuid: "3b728a70ee654948ab2fdf9f6d290630" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 41839 } } peers { permanent_uuid: "c241eb2dc27b4e96a82c4b3feade2d7f" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 43627 } }
I20260809 16:20:25.106494 24242 leader_election.cc:288] T 461a274e25f64897aab71acc64e0b9ff P c241eb2dc27b4e96a82c4b3feade2d7f [CANDIDATE]: Term 1 election: Requested vote from peers 30b6ddb95a86440f92bac5652b288162 (127.21.51.129:33979), 3b728a70ee654948ab2fdf9f6d290630 (127.21.51.130:41839)
I20260809 16:20:25.106653 23691 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "461a274e25f64897aab71acc64e0b9ff" candidate_uuid: "c241eb2dc27b4e96a82c4b3feade2d7f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30b6ddb95a86440f92bac5652b288162"
I20260809 16:20:25.106721 23912 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "461a274e25f64897aab71acc64e0b9ff" candidate_uuid: "c241eb2dc27b4e96a82c4b3feade2d7f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b728a70ee654948ab2fdf9f6d290630"
I20260809 16:20:25.106779 23691 raft_consensus.cc:3037] T 461a274e25f64897aab71acc64e0b9ff P 30b6ddb95a86440f92bac5652b288162 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:20:25.106787 23912 raft_consensus.cc:3037] T 461a274e25f64897aab71acc64e0b9ff P 3b728a70ee654948ab2fdf9f6d290630 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:20:25.107518 23912 raft_consensus.cc:2454] T 461a274e25f64897aab71acc64e0b9ff P 3b728a70ee654948ab2fdf9f6d290630 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c241eb2dc27b4e96a82c4b3feade2d7f in term 1.
I20260809 16:20:25.107524 23691 raft_consensus.cc:2454] T 461a274e25f64897aab71acc64e0b9ff P 30b6ddb95a86440f92bac5652b288162 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c241eb2dc27b4e96a82c4b3feade2d7f in term 1.
I20260809 16:20:25.107685 24021 leader_election.cc:302] T 461a274e25f64897aab71acc64e0b9ff P c241eb2dc27b4e96a82c4b3feade2d7f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b728a70ee654948ab2fdf9f6d290630, c241eb2dc27b4e96a82c4b3feade2d7f; no voters:
I20260809 16:20:25.107795 24242 raft_consensus.cc:2781] T 461a274e25f64897aab71acc64e0b9ff P c241eb2dc27b4e96a82c4b3feade2d7f [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:20:25.107852 24242 raft_consensus.cc:686] T 461a274e25f64897aab71acc64e0b9ff P c241eb2dc27b4e96a82c4b3feade2d7f [term 1 LEADER]: Becoming Leader. State: Replica: c241eb2dc27b4e96a82c4b3feade2d7f, State: Running, Role: LEADER
I20260809 16:20:25.107903 24242 consensus_queue.cc:237] T 461a274e25f64897aab71acc64e0b9ff P c241eb2dc27b4e96a82c4b3feade2d7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30b6ddb95a86440f92bac5652b288162" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 33979 } } peers { permanent_uuid: "3b728a70ee654948ab2fdf9f6d290630" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 41839 } } peers { permanent_uuid: "c241eb2dc27b4e96a82c4b3feade2d7f" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 43627 } }
I20260809 16:20:25.108285 24291 tablet_replica.cc:439] T 461a274e25f64897aab71acc64e0b9ff P c241eb2dc27b4e96a82c4b3feade2d7f: TxnStatusTablet state changed. Reason: New leader c241eb2dc27b4e96a82c4b3feade2d7f. Latest consensus state: current_term: 1 leader_uuid: "c241eb2dc27b4e96a82c4b3feade2d7f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30b6ddb95a86440f92bac5652b288162" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 33979 } } peers { permanent_uuid: "3b728a70ee654948ab2fdf9f6d290630" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 41839 } } peers { permanent_uuid: "c241eb2dc27b4e96a82c4b3feade2d7f" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 43627 } } }
I20260809 16:20:25.108410 24291 tablet_replica.cc:442] T 461a274e25f64897aab71acc64e0b9ff P c241eb2dc27b4e96a82c4b3feade2d7f: This TxnStatusTablet replica's current role is: LEADER
I20260809 16:20:25.108625 24312 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260809 16:20:25.108579 23436 catalog_manager.cc:5495] T 461a274e25f64897aab71acc64e0b9ff P c241eb2dc27b4e96a82c4b3feade2d7f reported cstate change: term changed from 0 to 1, leader changed from <none> to c241eb2dc27b4e96a82c4b3feade2d7f (127.21.51.131). New cstate: current_term: 1 leader_uuid: "c241eb2dc27b4e96a82c4b3feade2d7f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30b6ddb95a86440f92bac5652b288162" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 33979 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3b728a70ee654948ab2fdf9f6d290630" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 41839 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c241eb2dc27b4e96a82c4b3feade2d7f" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 43627 } health_report { overall_health: HEALTHY } } }
I20260809 16:20:25.108805 24312 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260809 16:20:25.140188 24223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:25.137617 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47084 (local address 127.21.51.131:43627)
0809 16:20:25.137660 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0809 16:20:25.137662 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:20:25.137710 (+ 48us) server_negotiation.cc:407] Connection header received
0809 16:20:25.137818 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:20:25.137819 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:20:25.137838 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:20:25.137899 (+ 61us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 16:20:25.138069 (+ 170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:25.138567 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:25.139346 (+ 779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:25.139566 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:25.139644 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:25.139838 (+ 194us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 16:20:25.139951 (+ 113us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 16:20:25.140036 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0809 16:20:25.140085 (+ 49us) server_negotiation.cc:299] Negotiation successful
0809 16:20:25.140112 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260809 16:20:25.140211 24304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:25.137582 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.131:43627 (local address 127.0.0.1:47084)
0809 16:20:25.137666 (+ 84us) negotiation.cc:107] Waiting for socket to connect
0809 16:20:25.137677 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 16:20:25.137733 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:20:25.137913 (+ 180us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:20:25.137914 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:20:25.137915 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 16:20:25.137971 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:20:25.137974 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:20:25.138654 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:20:25.138655 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:20:25.139248 (+ 593us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:20:25.139250 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:20:25.139646 (+ 396us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:20:25.139647 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:20:25.139716 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:25.139742 (+ 26us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 16:20:25.140022 (+ 280us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 16:20:25.140034 (+ 12us) client_negotiation.cc:769] Sending connection context
0809 16:20:25.140084 (+ 50us) client_negotiation.cc:240] Negotiation successful
0809 16:20:25.140140 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":49}
I20260809 16:20:25.141556 23691 raft_consensus.cc:1261] T 461a274e25f64897aab71acc64e0b9ff P 30b6ddb95a86440f92bac5652b288162 [term 1 FOLLOWER]: Refusing update from remote peer c241eb2dc27b4e96a82c4b3feade2d7f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 16:20:25.141697 23912 raft_consensus.cc:1261] T 461a274e25f64897aab71acc64e0b9ff P 3b728a70ee654948ab2fdf9f6d290630 [term 1 FOLLOWER]: Refusing update from remote peer c241eb2dc27b4e96a82c4b3feade2d7f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 16:20:25.141979 24242 consensus_queue.cc:1035] T 461a274e25f64897aab71acc64e0b9ff P c241eb2dc27b4e96a82c4b3feade2d7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "30b6ddb95a86440f92bac5652b288162" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 33979 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 16:20:25.142120 24291 consensus_queue.cc:1035] T 461a274e25f64897aab71acc64e0b9ff P c241eb2dc27b4e96a82c4b3feade2d7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b728a70ee654948ab2fdf9f6d290630" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 41839 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 16:20:25.142635 24238 tablet_replica.cc:439] T 461a274e25f64897aab71acc64e0b9ff P 30b6ddb95a86440f92bac5652b288162: TxnStatusTablet state changed. Reason: New leader c241eb2dc27b4e96a82c4b3feade2d7f. Latest consensus state: current_term: 1 leader_uuid: "c241eb2dc27b4e96a82c4b3feade2d7f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30b6ddb95a86440f92bac5652b288162" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 33979 } } peers { permanent_uuid: "3b728a70ee654948ab2fdf9f6d290630" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 41839 } } peers { permanent_uuid: "c241eb2dc27b4e96a82c4b3feade2d7f" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 43627 } } }
I20260809 16:20:25.142761 24238 tablet_replica.cc:442] T 461a274e25f64897aab71acc64e0b9ff P 30b6ddb95a86440f92bac5652b288162: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 16:20:25.142923 24261 tablet_replica.cc:439] T 461a274e25f64897aab71acc64e0b9ff P 3b728a70ee654948ab2fdf9f6d290630: TxnStatusTablet state changed. Reason: New leader c241eb2dc27b4e96a82c4b3feade2d7f. Latest consensus state: current_term: 1 leader_uuid: "c241eb2dc27b4e96a82c4b3feade2d7f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30b6ddb95a86440f92bac5652b288162" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 33979 } } peers { permanent_uuid: "3b728a70ee654948ab2fdf9f6d290630" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 41839 } } peers { permanent_uuid: "c241eb2dc27b4e96a82c4b3feade2d7f" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 43627 } } }
I20260809 16:20:25.143014 24261 tablet_replica.cc:442] T 461a274e25f64897aab71acc64e0b9ff P 3b728a70ee654948ab2fdf9f6d290630: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 16:20:25.143824 24291 tablet_replica.cc:439] T 461a274e25f64897aab71acc64e0b9ff P c241eb2dc27b4e96a82c4b3feade2d7f: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "c241eb2dc27b4e96a82c4b3feade2d7f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30b6ddb95a86440f92bac5652b288162" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 33979 } } peers { permanent_uuid: "3b728a70ee654948ab2fdf9f6d290630" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 41839 } } peers { permanent_uuid: "c241eb2dc27b4e96a82c4b3feade2d7f" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 43627 } } }
I20260809 16:20:25.143915 24291 tablet_replica.cc:442] T 461a274e25f64897aab71acc64e0b9ff P c241eb2dc27b4e96a82c4b3feade2d7f: This TxnStatusTablet replica's current role is: LEADER
I20260809 16:20:25.144126 24261 tablet_replica.cc:439] T 461a274e25f64897aab71acc64e0b9ff P 3b728a70ee654948ab2fdf9f6d290630: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c241eb2dc27b4e96a82c4b3feade2d7f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30b6ddb95a86440f92bac5652b288162" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 33979 } } peers { permanent_uuid: "3b728a70ee654948ab2fdf9f6d290630" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 41839 } } peers { permanent_uuid: "c241eb2dc27b4e96a82c4b3feade2d7f" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 43627 } } }
I20260809 16:20:25.144212 24261 tablet_replica.cc:442] T 461a274e25f64897aab71acc64e0b9ff P 3b728a70ee654948ab2fdf9f6d290630: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 16:20:25.145823 24242 tablet_replica.cc:439] T 461a274e25f64897aab71acc64e0b9ff P c241eb2dc27b4e96a82c4b3feade2d7f: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "c241eb2dc27b4e96a82c4b3feade2d7f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30b6ddb95a86440f92bac5652b288162" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 33979 } } peers { permanent_uuid: "3b728a70ee654948ab2fdf9f6d290630" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 41839 } } peers { permanent_uuid: "c241eb2dc27b4e96a82c4b3feade2d7f" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 43627 } } }
I20260809 16:20:25.145908 24242 tablet_replica.cc:442] T 461a274e25f64897aab71acc64e0b9ff P c241eb2dc27b4e96a82c4b3feade2d7f: This TxnStatusTablet replica's current role is: LEADER
I20260809 16:20:25.146561 24252 mvcc.cc:205] Tried to move back new op lower bound from 7316653773377646592 to 7316653773242351616. Current Snapshot: MvccSnapshot[applied={T|T < 7316653773377646592}]
I20260809 16:20:25.147123 24238 tablet_replica.cc:439] T 461a274e25f64897aab71acc64e0b9ff P 30b6ddb95a86440f92bac5652b288162: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c241eb2dc27b4e96a82c4b3feade2d7f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30b6ddb95a86440f92bac5652b288162" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 33979 } } peers { permanent_uuid: "3b728a70ee654948ab2fdf9f6d290630" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 41839 } } peers { permanent_uuid: "c241eb2dc27b4e96a82c4b3feade2d7f" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 43627 } } }
I20260809 16:20:25.147204 24238 tablet_replica.cc:442] T 461a274e25f64897aab71acc64e0b9ff P 30b6ddb95a86440f92bac5652b288162: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 16:20:25.148741 24223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:25.146047 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47092 (local address 127.21.51.131:43627)
0809 16:20:25.146086 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0809 16:20:25.146088 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:20:25.146096 (+ 8us) server_negotiation.cc:407] Connection header received
0809 16:20:25.146114 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:20:25.146115 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:20:25.146125 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:20:25.146174 (+ 49us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 16:20:25.146343 (+ 169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:25.147002 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:25.147997 (+ 995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:25.148249 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:25.148329 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:25.148483 (+ 154us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 16:20:25.148514 (+ 31us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 16:20:25.148561 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0809 16:20:25.148616 (+ 55us) server_negotiation.cc:299] Negotiation successful
0809 16:20:25.148635 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260809 16:20:25.153599 24223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:25.151161 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.51.131:36149 (local address 127.21.51.131:43627)
0809 16:20:25.151197 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0809 16:20:25.151199 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:20:25.151344 (+ 145us) server_negotiation.cc:407] Connection header received
0809 16:20:25.151397 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:20:25.151398 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:20:25.151407 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:20:25.151460 (+ 53us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 16:20:25.151574 (+ 114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:25.152098 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:25.152901 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:25.153093 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:25.153170 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:25.153356 (+ 186us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 16:20:25.153385 (+ 29us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 16:20:25.153435 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0809 16:20:25.153487 (+ 52us) server_negotiation.cc:299] Negotiation successful
0809 16:20:25.153504 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260809 16:20:25.153658 24323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:25.151113 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.131:43627 (local address 127.21.51.131:36149)
0809 16:20:25.151268 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0809 16:20:25.151278 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 16:20:25.151340 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:20:25.151463 (+ 123us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:20:25.151464 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:20:25.151465 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 16:20:25.151516 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:20:25.151517 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:20:25.152177 (+ 660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:20:25.152177 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:20:25.152808 (+ 631us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:20:25.152810 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:20:25.153169 (+ 359us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:20:25.153170 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:20:25.153241 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:25.153261 (+ 20us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 16:20:25.153441 (+ 180us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 16:20:25.153442 (+ 1us) client_negotiation.cc:769] Sending connection context
0809 16:20:25.153487 (+ 45us) client_negotiation.cc:240] Negotiation successful
0809 16:20:25.153527 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":107,"threads_started":1}
I20260809 16:20:25.156093 23912 raft_consensus.cc:1261] T 6367c459e7e1452abca2f541f4540cac P 3b728a70ee654948ab2fdf9f6d290630 [term 1 FOLLOWER]: Refusing update from remote peer c241eb2dc27b4e96a82c4b3feade2d7f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 16:20:25.156222 23691 raft_consensus.cc:1261] T 6367c459e7e1452abca2f541f4540cac P 30b6ddb95a86440f92bac5652b288162 [term 1 FOLLOWER]: Refusing update from remote peer c241eb2dc27b4e96a82c4b3feade2d7f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 16:20:25.156394 24291 consensus_queue.cc:1035] T 6367c459e7e1452abca2f541f4540cac P c241eb2dc27b4e96a82c4b3feade2d7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b728a70ee654948ab2fdf9f6d290630" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 41839 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 16:20:25.156471 24242 consensus_queue.cc:1035] T 6367c459e7e1452abca2f541f4540cac P c241eb2dc27b4e96a82c4b3feade2d7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "30b6ddb95a86440f92bac5652b288162" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 33979 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 16:20:25.172659 23435 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55066:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 16:20:25.172792 23435 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55066:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 16:20:25.173455 23435 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 052d5e814a1d4882854b5c6f215a148b: Sending DeleteTablet for 3 replicas of tablet 6367c459e7e1452abca2f541f4540cac
I20260809 16:20:25.173792 23669 tablet_service.cc:1514] Processing DeleteTablet for tablet 6367c459e7e1452abca2f541f4540cac with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:20:25 UTC) from {username='kudu', principal='kudu/127.21.51.190@KRBTEST.COM'} at 127.0.0.1:48936
I20260809 16:20:25.173878 24271 tablet_replica.cc:330] T 6367c459e7e1452abca2f541f4540cac P 30b6ddb95a86440f92bac5652b288162: stopping tablet replica
I20260809 16:20:25.173923 24271 raft_consensus.cc:2229] T 6367c459e7e1452abca2f541f4540cac P 30b6ddb95a86440f92bac5652b288162 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 16:20:25.173957 24271 raft_consensus.cc:2258] T 6367c459e7e1452abca2f541f4540cac P 30b6ddb95a86440f92bac5652b288162 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 16:20:25.174311 24271 ts_tablet_manager.cc:1905] T 6367c459e7e1452abca2f541f4540cac P 30b6ddb95a86440f92bac5652b288162: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 16:20:25.176043 24271 ts_tablet_manager.cc:1918] T 6367c459e7e1452abca2f541f4540cac P 30b6ddb95a86440f92bac5652b288162: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 16:20:25.176103 24271 log.cc:1196] T 6367c459e7e1452abca2f541f4540cac P 30b6ddb95a86440f92bac5652b288162: Deleting WAL directory at /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786292417722232-21710-0/minicluster-data/ts-0/wal/wals/6367c459e7e1452abca2f541f4540cac
I20260809 16:20:25.176214 23875 tablet_service.cc:1514] Processing DeleteTablet for tablet 6367c459e7e1452abca2f541f4540cac with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:20:25 UTC) from {username='kudu', principal='kudu/127.21.51.190@KRBTEST.COM'} at 127.0.0.1:37706
I20260809 16:20:25.176317 24278 tablet_replica.cc:330] T 6367c459e7e1452abca2f541f4540cac P 3b728a70ee654948ab2fdf9f6d290630: stopping tablet replica
I20260809 16:20:25.176342 24271 ts_tablet_manager.cc:1939] T 6367c459e7e1452abca2f541f4540cac P 30b6ddb95a86440f92bac5652b288162: Deleting consensus metadata
I20260809 16:20:25.176371 24278 raft_consensus.cc:2229] T 6367c459e7e1452abca2f541f4540cac P 3b728a70ee654948ab2fdf9f6d290630 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 16:20:25.176415 24278 raft_consensus.cc:2258] T 6367c459e7e1452abca2f541f4540cac P 3b728a70ee654948ab2fdf9f6d290630 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 16:20:25.176828 23413 catalog_manager.cc:4841] TS 30b6ddb95a86440f92bac5652b288162 (127.21.51.129:33979): tablet 6367c459e7e1452abca2f541f4540cac (table test-table [id=66982997de8a429296f33083c2777d5f]) successfully deleted
I20260809 16:20:25.177542 24099 tablet_service.cc:1514] Processing DeleteTablet for tablet 6367c459e7e1452abca2f541f4540cac with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:20:25 UTC) from {username='kudu', principal='kudu/127.21.51.190@KRBTEST.COM'} at 127.0.0.1:47068
I20260809 16:20:25.177655 24274 tablet_replica.cc:330] T 6367c459e7e1452abca2f541f4540cac P c241eb2dc27b4e96a82c4b3feade2d7f: stopping tablet replica
I20260809 16:20:25.177722 24274 raft_consensus.cc:2229] T 6367c459e7e1452abca2f541f4540cac P c241eb2dc27b4e96a82c4b3feade2d7f [term 1 LEADER]: Raft consensus shutting down.
I20260809 16:20:25.177822 24274 raft_consensus.cc:2258] T 6367c459e7e1452abca2f541f4540cac P c241eb2dc27b4e96a82c4b3feade2d7f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 16:20:25.177935 24278 ts_tablet_manager.cc:1905] T 6367c459e7e1452abca2f541f4540cac P 3b728a70ee654948ab2fdf9f6d290630: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 16:20:25.178141 24274 ts_tablet_manager.cc:1905] T 6367c459e7e1452abca2f541f4540cac P c241eb2dc27b4e96a82c4b3feade2d7f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 16:20:25.179646 24278 ts_tablet_manager.cc:1918] T 6367c459e7e1452abca2f541f4540cac P 3b728a70ee654948ab2fdf9f6d290630: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 16:20:25.179705 24278 log.cc:1196] T 6367c459e7e1452abca2f541f4540cac P 3b728a70ee654948ab2fdf9f6d290630: Deleting WAL directory at /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786292417722232-21710-0/minicluster-data/ts-1/wal/wals/6367c459e7e1452abca2f541f4540cac
I20260809 16:20:25.179934 24278 ts_tablet_manager.cc:1939] T 6367c459e7e1452abca2f541f4540cac P 3b728a70ee654948ab2fdf9f6d290630: Deleting consensus metadata
I20260809 16:20:25.180084 24274 ts_tablet_manager.cc:1918] T 6367c459e7e1452abca2f541f4540cac P c241eb2dc27b4e96a82c4b3feade2d7f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 16:20:25.180143 24274 log.cc:1196] T 6367c459e7e1452abca2f541f4540cac P c241eb2dc27b4e96a82c4b3feade2d7f: Deleting WAL directory at /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786292417722232-21710-0/minicluster-data/ts-2/wal/wals/6367c459e7e1452abca2f541f4540cac
I20260809 16:20:25.180267 23411 catalog_manager.cc:4841] TS 3b728a70ee654948ab2fdf9f6d290630 (127.21.51.130:41839): tablet 6367c459e7e1452abca2f541f4540cac (table test-table [id=66982997de8a429296f33083c2777d5f]) successfully deleted
I20260809 16:20:25.180383 24274 ts_tablet_manager.cc:1939] T 6367c459e7e1452abca2f541f4540cac P c241eb2dc27b4e96a82c4b3feade2d7f: Deleting consensus metadata
I20260809 16:20:25.180752 23414 catalog_manager.cc:4841] TS c241eb2dc27b4e96a82c4b3feade2d7f (127.21.51.131:43627): tablet 6367c459e7e1452abca2f541f4540cac (table test-table [id=66982997de8a429296f33083c2777d5f]) successfully deleted
Aug 09 16:20:25 dist-test-slave-s11t krb5kdc[23361](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292424, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.21.51.129@KRBTEST.COM
I20260809 16:20:25.187351 24220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:25.177681 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48944 (local address 127.21.51.129:33979)
0809 16:20:25.177722 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0809 16:20:25.177724 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:20:25.178000 (+ 276us) server_negotiation.cc:407] Connection header received
0809 16:20:25.178101 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:20:25.178102 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:20:25.178117 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:20:25.178149 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:20:25.178915 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:25.179643 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:25.180931 (+ 1288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:25.181155 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:25.181259 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:25.184500 (+ 3241us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:20:25.184510 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:20:25.184511 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:20:25.184518 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:20:25.186139 (+ 1621us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:20:25.186506 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:20:25.186509 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:20:25.186510 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:20:25.186578 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:20:25.186797 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:20:25.186798 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:20:25.186798 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:20:25.186941 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:20:25.187088 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0809 16:20:25.187166 (+ 78us) server_negotiation.cc:299] Negotiation successful
0809 16:20:25.187251 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
W20260809 16:20:25.187722 23740 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:48944
I20260809 16:20:25.197944 24306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:25.188621 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45514 (local address 127.21.51.190:33935)
0809 16:20:25.188664 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0809 16:20:25.188666 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:20:25.188849 (+ 183us) server_negotiation.cc:407] Connection header received
0809 16:20:25.188921 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:20:25.188921 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:20:25.188932 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:20:25.188987 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:20:25.189480 (+ 493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:25.189978 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:25.190662 (+ 684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:25.190888 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:25.190966 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:25.195199 (+ 4233us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:20:25.195200 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:20:25.195201 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:20:25.195209 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:20:25.196797 (+ 1588us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:20:25.197148 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:20:25.197150 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:20:25.197150 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:20:25.197213 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:20:25.197419 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:20:25.197420 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:20:25.197421 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:20:25.197542 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:20:25.197648 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0809 16:20:25.197748 (+ 100us) server_negotiation.cc:299] Negotiation successful
0809 16:20:25.197845 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
W20260809 16:20:25.198267 23435 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45514
I20260809 16:20:25.198824 21710 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrNqUCn/build/release/bin/kudu with pid 23554
I20260809 16:20:25.211700 21710 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrNqUCn/build/release/bin/kudu with pid 23777
I20260809 16:20:25.222122 21710 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrNqUCn/build/release/bin/kudu with pid 23990
I20260809 16:20:25.231791 21710 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrNqUCn/build/release/bin/kudu with pid 23385
2026-08-09T16:20:25Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (2931 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786292417722232-21710-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 16:20:25 dist-test-slave-s11t krb5kdc[24339](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 16:20:25 dist-test-slave-s11t krb5kdc[24339](info): set up 2 sockets
Aug 09 16:20:25 dist-test-slave-s11t krb5kdc[24339](info): commencing operation
krb5kdc: starting...
W20260809 16:20:26.487881 21710 mini_kdc.cc:121] Time spent starting KDC: real 1.225s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786292417722232-21710-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786292417722232-21710-0/krb5kdc/test-user.keytab.
Aug 09 16:20:26 dist-test-slave-s11t krb5kdc[24339](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292426, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T16:20:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T16:20:26Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.51.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.51.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.51.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.190.keytab.
Entry for principal kudu/127.21.51.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.190.keytab.
I20260809 16:20:26.615530 21710 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrNqUCn/build/release/bin/kudu
/tmp/dist-test-taskrNqUCn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786292417722232-21710-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786292417722232-21710-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786292417722232-21710-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786292417722232-21710-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.51.190:38211
--webserver_interface=127.21.51.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.190.keytab
--principal=kudu/127.21.51.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.51.148:36335
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.51.190:38211
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786292417722232-21710-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786292417722232-21710-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786292417722232-21710-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:20:26.698988 24357 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:20:26.699218 24357 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:20:26.699257 24357 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:20:26.700395 24357 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 16:20:26.700461 24357 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260809 16:20:26.700485 24357 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:20:26.700517 24357 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:20:26.700544 24357 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 16:20:26.700567 24357 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 16:20:26.702112 24357 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.51.148:36335
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786292417722232-21710-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786292417722232-21710-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.51.190:38211
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.190.keytab
--principal=kudu/127.21.51.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.51.190:38211
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786292417722232-21710-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.51.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24357
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786292417722232-21710-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:20:26.702402 24357 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:20:26.702648 24357 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:20:26.707824 24364 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:20:26.707881 24363 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:20:26.707780 24366 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:20:26.707995 24357 server_base.cc:1034] running on GCE node
I20260809 16:20:26.708274 24357 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:20:26.708527 24357 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:20:26.709697 24357 hybrid_clock.cc:648] HybridClock initialized: now 1786292426709669 us; error 39 us; skew 500 ppm
Aug 09 16:20:26 dist-test-slave-s11t krb5kdc[24339](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292426, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.51.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:20:26.712352 24357 init.cc:377] Logged in from keytab as kudu/127.21.51.190@KRBTEST.COM (short username kudu)
I20260809 16:20:26.712914 24357 webserver.cc:460] Webserver started at http://127.21.51.190:46615/ using document root <none> and password file <none>
I20260809 16:20:26.713164 24357 fs_manager.cc:359] Metadata directory not provided
I20260809 16:20:26.713234 24357 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:20:26.713392 24357 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:20:26.714500 24357 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786292417722232-21710-0/minicluster-data/master-0/data/instance:
uuid: "4d1cbb0203184af5a564b1a0deb4dd3f"
format_stamp: "Formatted at 2026-08-09 16:20:26 on dist-test-slave-s11t"
server_key: "8c2140f5062c05de1c0726716ffa2c53"
server_key_iv: "3cf2387683668a136aa1f42440243142"
server_key_version: "encryptionkey@0"
I20260809 16:20:26.714872 24357 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786292417722232-21710-0/minicluster-data/master-0/wal/instance:
uuid: "4d1cbb0203184af5a564b1a0deb4dd3f"
format_stamp: "Formatted at 2026-08-09 16:20:26 on dist-test-slave-s11t"
server_key: "8c2140f5062c05de1c0726716ffa2c53"
server_key_iv: "3cf2387683668a136aa1f42440243142"
server_key_version: "encryptionkey@0"
I20260809 16:20:26.716233 24357 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 16:20:26.717060 24382 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:20:26.717238 24357 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 16:20:26.717360 24357 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786292417722232-21710-0/minicluster-data/master-0/data,/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786292417722232-21710-0/minicluster-data/master-0/wal
uuid: "4d1cbb0203184af5a564b1a0deb4dd3f"
format_stamp: "Formatted at 2026-08-09 16:20:26 on dist-test-slave-s11t"
server_key: "8c2140f5062c05de1c0726716ffa2c53"
server_key_iv: "3cf2387683668a136aa1f42440243142"
server_key_version: "encryptionkey@0"
I20260809 16:20:26.717444 24357 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786292417722232-21710-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786292417722232-21710-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786292417722232-21710-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:20:26.746927 24357 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:20:26.749043 24357 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:20:26.749213 24357 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:20:26.753961 24357 rpc_server.cc:307] RPC server started. Bound to: 127.21.51.190:38211
I20260809 16:20:26.754014 24479 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.51.190:38211 every 8 connection(s)
I20260809 16:20:26.754354 24357 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786292417722232-21710-0/minicluster-data/master-0/data/info.pb
I20260809 16:20:26.755004 24481 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:20:26.757427 24481 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4d1cbb0203184af5a564b1a0deb4dd3f: Bootstrap starting.
I20260809 16:20:26.758045 24481 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4d1cbb0203184af5a564b1a0deb4dd3f: Neither blocks nor log segments found. Creating new log.
I20260809 16:20:26.758334 24481 log.cc:824] T 00000000000000000000000000000000 P 4d1cbb0203184af5a564b1a0deb4dd3f: Log is configured to *not* fsync() on all Append() calls
I20260809 16:20:26.759038 24481 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4d1cbb0203184af5a564b1a0deb4dd3f: No bootstrap required, opened a new log
I20260809 16:20:26.761121 24481 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4d1cbb0203184af5a564b1a0deb4dd3f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d1cbb0203184af5a564b1a0deb4dd3f" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 38211 } }
I20260809 16:20:26.761288 24481 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4d1cbb0203184af5a564b1a0deb4dd3f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:20:26.761329 24481 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4d1cbb0203184af5a564b1a0deb4dd3f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4d1cbb0203184af5a564b1a0deb4dd3f, State: Initialized, Role: FOLLOWER
I20260809 16:20:26.761541 21710 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrNqUCn/build/release/bin/kudu as pid 24357
I20260809 16:20:26.761510 24481 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4d1cbb0203184af5a564b1a0deb4dd3f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d1cbb0203184af5a564b1a0deb4dd3f" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 38211 } }
I20260809 16:20:26.761612 24481 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4d1cbb0203184af5a564b1a0deb4dd3f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 16:20:26.761616 21710 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786292417722232-21710-0/minicluster-data/master-0/wal/instance
I20260809 16:20:26.761656 24481 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4d1cbb0203184af5a564b1a0deb4dd3f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 16:20:26.761710 24481 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4d1cbb0203184af5a564b1a0deb4dd3f [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:20:26.761826 21710 external_mini_cluster.cc:1428] Setting key a60b6adf2c062ff4362d0c5b45d00679
I20260809 16:20:26.762362 24481 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4d1cbb0203184af5a564b1a0deb4dd3f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d1cbb0203184af5a564b1a0deb4dd3f" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 38211 } }
I20260809 16:20:26.762488 24481 leader_election.cc:302] T 00000000000000000000000000000000 P 4d1cbb0203184af5a564b1a0deb4dd3f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4d1cbb0203184af5a564b1a0deb4dd3f; no voters:
I20260809 16:20:26.762657 24481 leader_election.cc:288] T 00000000000000000000000000000000 P 4d1cbb0203184af5a564b1a0deb4dd3f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 16:20:26.762702 24489 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4d1cbb0203184af5a564b1a0deb4dd3f [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:20:26.762810 24489 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4d1cbb0203184af5a564b1a0deb4dd3f [term 1 LEADER]: Becoming Leader. State: Replica: 4d1cbb0203184af5a564b1a0deb4dd3f, State: Running, Role: LEADER
I20260809 16:20:26.762933 24481 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4d1cbb0203184af5a564b1a0deb4dd3f [sys.catalog]: configured and running, proceeding with master startup.
I20260809 16:20:26.762893 24489 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4d1cbb0203184af5a564b1a0deb4dd3f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d1cbb0203184af5a564b1a0deb4dd3f" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 38211 } }
I20260809 16:20:26.763286 24491 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4d1cbb0203184af5a564b1a0deb4dd3f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4d1cbb0203184af5a564b1a0deb4dd3f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d1cbb0203184af5a564b1a0deb4dd3f" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 38211 } } }
I20260809 16:20:26.763411 24491 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4d1cbb0203184af5a564b1a0deb4dd3f [sys.catalog]: This master's current role is: LEADER
I20260809 16:20:26.763273 24490 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4d1cbb0203184af5a564b1a0deb4dd3f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4d1cbb0203184af5a564b1a0deb4dd3f. Latest consensus state: current_term: 1 leader_uuid: "4d1cbb0203184af5a564b1a0deb4dd3f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d1cbb0203184af5a564b1a0deb4dd3f" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 38211 } } }
I20260809 16:20:26.763670 24490 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4d1cbb0203184af5a564b1a0deb4dd3f [sys.catalog]: This master's current role is: LEADER
I20260809 16:20:26.764086 24498 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 16:20:26.764601 24498 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 16:20:26.766242 24498 catalog_manager.cc:1347] Generated new cluster ID: 3b43416ab9014beca81e1f9980c6fb30
I20260809 16:20:26.766299 24498 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 09 16:20:26 dist-test-slave-s11t krb5kdc[24339](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292426, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.51.190@KRBTEST.COM
I20260809 16:20:26.772927 24488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:26.762340 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60106 (local address 127.21.51.190:38211)
0809 16:20:26.762533 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0809 16:20:26.762543 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 16:20:26.762580 (+ 37us) server_negotiation.cc:407] Connection header received
0809 16:20:26.762699 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:20:26.762705 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:20:26.762780 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:20:26.762961 (+ 181us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:20:26.764118 (+ 1157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:26.765560 (+ 1442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:26.766612 (+ 1052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:26.767044 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:26.767169 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:26.769557 (+ 2388us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:20:26.769561 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:20:26.769561 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:20:26.769574 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:20:26.771335 (+ 1761us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:20:26.771724 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:20:26.771727 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:20:26.771727 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:20:26.771835 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:20:26.772058 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:20:26.772059 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:20:26.772060 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:20:26.772306 (+ 246us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:20:26.772471 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0809 16:20:26.772590 (+ 119us) server_negotiation.cc:299] Negotiation successful
0809 16:20:26.772760 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":138,"thread_start_us":86,"threads_started":1}
I20260809 16:20:26.807030 24498 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 16:20:26.807672 24498 catalog_manager.cc:1504] Loading token signing keys...
I20260809 16:20:26.813167 24498 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4d1cbb0203184af5a564b1a0deb4dd3f: Generated new TSK 0
I20260809 16:20:26.813374 24498 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.21.51.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.51.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.51.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.129.keytab.
Entry for principal kudu/127.21.51.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.129.keytab.
I20260809 16:20:26.849999 21710 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrNqUCn/build/release/bin/kudu
/tmp/dist-test-taskrNqUCn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786292417722232-21710-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786292417722232-21710-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786292417722232-21710-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786292417722232-21710-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.51.129:0
--local_ip_for_outbound_sockets=127.21.51.129
--webserver_interface=127.21.51.129
--webserver_port=0
--tserver_master_addrs=127.21.51.190:38211
--keytab_file=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.129.keytab
--principal=kudu/127.21.51.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.51.148:36335
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786292417722232-21710-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786292417722232-21710-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786292417722232-21710-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:20:26.933980 24518 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:20:26.934183 24518 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:20:26.934207 24518 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:20:26.935408 24518 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:20:26.935451 24518 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:20:26.935472 24518 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260809 16:20:26.935483 24518 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 16:20:26.935539 24518 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.51.129
I20260809 16:20:26.936966 24518 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.51.148:36335
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786292417722232-21710-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786292417722232-21710-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.129.keytab
--principal=kudu/127.21.51.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.51.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786292417722232-21710-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.51.129
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.21.51.190:38211
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24518
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.51.129
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786292417722232-21710-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:20:26.937211 24518 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:20:26.937453 24518 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:20:26.942976 24523 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:20:26.943022 24524 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:20:26.942950 24527 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:20:26.943159 24518 server_base.cc:1034] running on GCE node
I20260809 16:20:26.943392 24518 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:20:26.943686 24518 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:20:26.944875 24518 hybrid_clock.cc:648] HybridClock initialized: now 1786292426944861 us; error 30 us; skew 500 ppm
Aug 09 16:20:26 dist-test-slave-s11t krb5kdc[24339](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292426, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.51.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:20:26.947415 24518 init.cc:377] Logged in from keytab as kudu/127.21.51.129@KRBTEST.COM (short username kudu)
I20260809 16:20:26.947876 24518 webserver.cc:460] Webserver started at http://127.21.51.129:36983/ using document root <none> and password file <none>
I20260809 16:20:26.948056 24518 fs_manager.cc:359] Metadata directory not provided
I20260809 16:20:26.948096 24518 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:20:26.948184 24518 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:20:26.949239 24518 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786292417722232-21710-0/minicluster-data/ts-0/data/instance:
uuid: "4f1796cdf8d4421fb7f9d0f614743bdc"
format_stamp: "Formatted at 2026-08-09 16:20:26 on dist-test-slave-s11t"
server_key: "e561c170efa45e10e06eec560866a23a"
server_key_iv: "ecd951aae1451f6e0ce010cd977746ac"
server_key_version: "encryptionkey@0"
I20260809 16:20:26.949542 24518 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786292417722232-21710-0/minicluster-data/ts-0/wal/instance:
uuid: "4f1796cdf8d4421fb7f9d0f614743bdc"
format_stamp: "Formatted at 2026-08-09 16:20:26 on dist-test-slave-s11t"
server_key: "e561c170efa45e10e06eec560866a23a"
server_key_iv: "ecd951aae1451f6e0ce010cd977746ac"
server_key_version: "encryptionkey@0"
I20260809 16:20:26.950857 24518 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 16:20:26.951561 24535 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:20:26.951768 24518 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:20:26.951857 24518 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786292417722232-21710-0/minicluster-data/ts-0/data,/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786292417722232-21710-0/minicluster-data/ts-0/wal
uuid: "4f1796cdf8d4421fb7f9d0f614743bdc"
format_stamp: "Formatted at 2026-08-09 16:20:26 on dist-test-slave-s11t"
server_key: "e561c170efa45e10e06eec560866a23a"
server_key_iv: "ecd951aae1451f6e0ce010cd977746ac"
server_key_version: "encryptionkey@0"
I20260809 16:20:26.951911 24518 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786292417722232-21710-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786292417722232-21710-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786292417722232-21710-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:20:26.958254 24518 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:20:26.960074 24518 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:20:26.960182 24518 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:20:26.960616 24518 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:20:26.960652 24518 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:20:26.960677 24518 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:20:26.960757 24518 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:20:26.967653 24518 rpc_server.cc:307] RPC server started. Bound to: 127.21.51.129:39087
I20260809 16:20:26.967722 24719 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.51.129:39087 every 8 connection(s)
Aug 09 16:20:26 dist-test-slave-s11t krb5kdc[24339](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292426, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.51.129@KRBTEST.COM for kudu/127.21.51.190@KRBTEST.COM
I20260809 16:20:26.968056 24518 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786292417722232-21710-0/minicluster-data/ts-0/data/info.pb
I20260809 16:20:26.971359 24488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:26.961180 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.51.129:54811 (local address 127.21.51.190:38211)
0809 16:20:26.961255 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0809 16:20:26.961258 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 16:20:26.961392 (+ 134us) server_negotiation.cc:407] Connection header received
0809 16:20:26.961840 (+ 448us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:20:26.961841 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:20:26.961857 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:20:26.961939 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:20:26.962878 (+ 939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:26.963713 (+ 835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:26.965337 (+ 1624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:26.965689 (+ 352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:26.965776 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:26.968432 (+ 2656us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:20:26.968434 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:20:26.968434 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:20:26.968443 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:20:26.969951 (+ 1508us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:20:26.970368 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:20:26.970369 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:20:26.970370 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:20:26.970436 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:20:26.970696 (+ 260us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:20:26.970697 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:20:26.970698 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:20:26.970820 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:20:26.970894 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0809 16:20:26.971172 (+ 278us) server_negotiation.cc:299] Negotiation successful
0809 16:20:26.971260 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260809 16:20:26.971517 24550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:26.961096 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:38211 (local address 127.21.51.129:54811)
0809 16:20:26.961267 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0809 16:20:26.961306 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0809 16:20:26.961744 (+ 438us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:20:26.962075 (+ 331us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:20:26.962080 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:20:26.962437 (+ 357us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:20:26.962768 (+ 331us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:20:26.962774 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:20:26.963852 (+ 1078us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:20:26.963853 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:20:26.965243 (+ 1390us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:20:26.965246 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:20:26.965791 (+ 545us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:20:26.965792 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:20:26.965927 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:26.966413 (+ 486us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:20:26.966442 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:20:26.968343 (+ 1901us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:20:26.970059 (+ 1716us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:20:26.970067 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:20:26.970069 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:20:26.970288 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:20:26.970506 (+ 218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:20:26.970506 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:20:26.970507 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:20:26.970619 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:20:26.970885 (+ 266us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:20:26.970892 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:20:26.971029 (+ 137us) client_negotiation.cc:769] Sending connection context
0809 16:20:26.971134 (+ 105us) client_negotiation.cc:240] Negotiation successful
0809 16:20:26.971292 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":106,"threads_started":1}
I20260809 16:20:26.972026 24720 heartbeater.cc:347] Connected to a master server at 127.21.51.190:38211
I20260809 16:20:26.972116 24720 heartbeater.cc:464] Registering TS with master...
I20260809 16:20:26.972314 24720 heartbeater.cc:511] Master 127.21.51.190:38211 requested a full tablet report, sending...
I20260809 16:20:26.972942 24410 ts_manager.cc:194] Registered new tserver with Master: 4f1796cdf8d4421fb7f9d0f614743bdc (127.21.51.129:39087)
I20260809 16:20:26.973656 24410 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.51.129@KRBTEST.COM'} at 127.21.51.129:54811
I20260809 16:20:26.975440 21710 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrNqUCn/build/release/bin/kudu as pid 24518
I20260809 16:20:26.975526 21710 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786292417722232-21710-0/minicluster-data/ts-0/wal/instance
I20260809 16:20:26.975729 21710 external_mini_cluster.cc:1428] Setting key cf4beb5ac58e743aca44c67c224c8810
I20260809 16:20:26.979511 24488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:26.973180 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.51.129:43015 (local address 127.21.51.190:38211)
0809 16:20:26.973216 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0809 16:20:26.973218 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:20:26.973356 (+ 138us) server_negotiation.cc:407] Connection header received
0809 16:20:26.973430 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:20:26.973431 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:20:26.973441 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:20:26.973489 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:20:26.973911 (+ 422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:26.974597 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:26.975293 (+ 696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:26.975588 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:26.975660 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:26.976714 (+ 1054us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:20:26.976715 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:20:26.976716 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:20:26.976723 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:20:26.978282 (+ 1559us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:20:26.978715 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:20:26.978717 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:20:26.978717 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:20:26.978783 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:20:26.978992 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:20:26.978993 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:20:26.978993 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:20:26.979103 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:20:26.979181 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0809 16:20:26.979343 (+ 162us) server_negotiation.cc:299] Negotiation successful
0809 16:20:26.979423 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260809 16:20:26.979527 24726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:26.973141 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:38211 (local address 127.21.51.129:43015)
0809 16:20:26.973275 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0809 16:20:26.973286 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 16:20:26.973371 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:20:26.973492 (+ 121us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:20:26.973492 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:20:26.973667 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:20:26.973752 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:20:26.973754 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:20:26.974688 (+ 934us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:20:26.974689 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:20:26.975192 (+ 503us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:20:26.975193 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:20:26.975686 (+ 493us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:20:26.975687 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:20:26.975754 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:26.976216 (+ 462us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:20:26.976221 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:20:26.976609 (+ 388us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:20:26.978394 (+ 1785us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:20:26.978394 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:20:26.978395 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:20:26.978610 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:20:26.978865 (+ 255us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:20:26.978866 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:20:26.978866 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:20:26.978927 (+ 61us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:20:26.979172 (+ 245us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:20:26.979173 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:20:26.979231 (+ 58us) client_negotiation.cc:769] Sending connection context
0809 16:20:26.979317 (+ 86us) client_negotiation.cc:240] Negotiation successful
0809 16:20:26.979425 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":81,"threads_started":1}
WARNING: no policy specified for kudu/127.21.51.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.51.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.51.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.130.keytab.
Entry for principal kudu/127.21.51.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.130.keytab.
I20260809 16:20:27.003839 21710 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrNqUCn/build/release/bin/kudu
/tmp/dist-test-taskrNqUCn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786292417722232-21710-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786292417722232-21710-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786292417722232-21710-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786292417722232-21710-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.51.130:0
--local_ip_for_outbound_sockets=127.21.51.130
--webserver_interface=127.21.51.130
--webserver_port=0
--tserver_master_addrs=127.21.51.190:38211
--keytab_file=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.130.keytab
--principal=kudu/127.21.51.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.51.148:36335
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786292417722232-21710-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786292417722232-21710-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786292417722232-21710-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:20:27.087728 24733 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:20:27.087939 24733 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:20:27.087963 24733 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:20:27.089212 24733 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:20:27.089258 24733 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:20:27.089279 24733 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260809 16:20:27.089291 24733 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 16:20:27.089309 24733 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.51.130
I20260809 16:20:27.090763 24733 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.51.148:36335
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786292417722232-21710-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786292417722232-21710-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.130.keytab
--principal=kudu/127.21.51.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.51.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786292417722232-21710-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.51.130
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.21.51.190:38211
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24733
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.51.130
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786292417722232-21710-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:20:27.091009 24733 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:20:27.091262 24733 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:20:27.096349 24742 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:20:27.096386 24748 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:20:27.096354 24745 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:20:27.096632 24733 server_base.cc:1034] running on GCE node
I20260809 16:20:27.096884 24733 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:20:27.097141 24733 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:20:27.098359 24733 hybrid_clock.cc:648] HybridClock initialized: now 1786292427098335 us; error 39 us; skew 500 ppm
Aug 09 16:20:27 dist-test-slave-s11t krb5kdc[24339](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292427, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.51.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:20:27.101370 24733 init.cc:377] Logged in from keytab as kudu/127.21.51.130@KRBTEST.COM (short username kudu)
I20260809 16:20:27.101845 24733 webserver.cc:460] Webserver started at http://127.21.51.130:34821/ using document root <none> and password file <none>
I20260809 16:20:27.102111 24733 fs_manager.cc:359] Metadata directory not provided
I20260809 16:20:27.102187 24733 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:20:27.102358 24733 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:20:27.103511 24733 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786292417722232-21710-0/minicluster-data/ts-1/data/instance:
uuid: "18fb7927ff43460790088c7b6465d2fb"
format_stamp: "Formatted at 2026-08-09 16:20:27 on dist-test-slave-s11t"
server_key: "0303cec512403e058ad201866bf1c75a"
server_key_iv: "7046eaf6414446bf1184f5266e922523"
server_key_version: "encryptionkey@0"
I20260809 16:20:27.103917 24733 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786292417722232-21710-0/minicluster-data/ts-1/wal/instance:
uuid: "18fb7927ff43460790088c7b6465d2fb"
format_stamp: "Formatted at 2026-08-09 16:20:27 on dist-test-slave-s11t"
server_key: "0303cec512403e058ad201866bf1c75a"
server_key_iv: "7046eaf6414446bf1184f5266e922523"
server_key_version: "encryptionkey@0"
I20260809 16:20:27.105603 24733 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 16:20:27.106382 24758 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:20:27.106556 24733 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 16:20:27.106664 24733 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786292417722232-21710-0/minicluster-data/ts-1/data,/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786292417722232-21710-0/minicluster-data/ts-1/wal
uuid: "18fb7927ff43460790088c7b6465d2fb"
format_stamp: "Formatted at 2026-08-09 16:20:27 on dist-test-slave-s11t"
server_key: "0303cec512403e058ad201866bf1c75a"
server_key_iv: "7046eaf6414446bf1184f5266e922523"
server_key_version: "encryptionkey@0"
I20260809 16:20:27.106740 24733 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786292417722232-21710-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786292417722232-21710-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786292417722232-21710-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:20:27.131851 24733 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:20:27.134023 24733 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:20:27.134224 24733 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:20:27.134752 24733 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:20:27.134816 24733 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:20:27.134868 24733 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:20:27.134898 24733 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 16:20:27 dist-test-slave-s11t krb5kdc[24339](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292427, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.51.130@KRBTEST.COM for kudu/127.21.51.190@KRBTEST.COM
I20260809 16:20:27.142342 24733 rpc_server.cc:307] RPC server started. Bound to: 127.21.51.130:43831
I20260809 16:20:27.142710 24733 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786292417722232-21710-0/minicluster-data/ts-1/data/info.pb
I20260809 16:20:27.143152 24942 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.51.130:43831 every 8 connection(s)
I20260809 16:20:27.145092 24488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:27.135338 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.51.130:44899 (local address 127.21.51.190:38211)
0809 16:20:27.135417 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0809 16:20:27.135420 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 16:20:27.135602 (+ 182us) server_negotiation.cc:407] Connection header received
0809 16:20:27.136125 (+ 523us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:20:27.136127 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:20:27.136139 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:20:27.136202 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:20:27.137022 (+ 820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:27.137523 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:27.138626 (+ 1103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:27.138869 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:27.138946 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:27.141824 (+ 2878us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:20:27.141825 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:20:27.141825 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:20:27.141834 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:20:27.143562 (+ 1728us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:20:27.144040 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:20:27.144042 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:20:27.144043 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:20:27.144104 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:20:27.144373 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:20:27.144373 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:20:27.144374 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:20:27.144492 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:20:27.144576 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0809 16:20:27.144908 (+ 332us) server_negotiation.cc:299] Negotiation successful
0809 16:20:27.145002 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260809 16:20:27.145274 24768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:27.135303 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:38211 (local address 127.21.51.130:44899)
0809 16:20:27.135484 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0809 16:20:27.135523 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0809 16:20:27.136013 (+ 490us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:20:27.136284 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:20:27.136288 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:20:27.136677 (+ 389us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:20:27.136916 (+ 239us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:20:27.136921 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:20:27.137624 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:20:27.137625 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:20:27.138528 (+ 903us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:20:27.138530 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:20:27.138958 (+ 428us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:20:27.138959 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:20:27.139135 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:27.139787 (+ 652us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:20:27.139798 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:20:27.141736 (+ 1938us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:20:27.143660 (+ 1924us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:20:27.143667 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:20:27.143669 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:20:27.143947 (+ 278us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:20:27.144179 (+ 232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:20:27.144180 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:20:27.144180 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:20:27.144296 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:20:27.144566 (+ 270us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:20:27.144573 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:20:27.144763 (+ 190us) client_negotiation.cc:769] Sending connection context
0809 16:20:27.144882 (+ 119us) client_negotiation.cc:240] Negotiation successful
0809 16:20:27.145036 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":107,"threads_started":1}
I20260809 16:20:27.145763 24943 heartbeater.cc:347] Connected to a master server at 127.21.51.190:38211
I20260809 16:20:27.145874 24943 heartbeater.cc:464] Registering TS with master...
I20260809 16:20:27.146047 24943 heartbeater.cc:511] Master 127.21.51.190:38211 requested a full tablet report, sending...
I20260809 16:20:27.146466 24411 ts_manager.cc:194] Registered new tserver with Master: 18fb7927ff43460790088c7b6465d2fb (127.21.51.130:43831)
I20260809 16:20:27.147051 24411 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.51.130@KRBTEST.COM'} at 127.21.51.130:44899
I20260809 16:20:27.149432 21710 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrNqUCn/build/release/bin/kudu as pid 24733
I20260809 16:20:27.149545 21710 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786292417722232-21710-0/minicluster-data/ts-1/wal/instance
I20260809 16:20:27.149724 21710 external_mini_cluster.cc:1428] Setting key 2929e4ef386a142fa0f82bac41dbed70
I20260809 16:20:27.152988 24488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:27.146842 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.51.130:56945 (local address 127.21.51.190:38211)
0809 16:20:27.146884 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0809 16:20:27.146886 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:20:27.147008 (+ 122us) server_negotiation.cc:407] Connection header received
0809 16:20:27.147048 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:20:27.147048 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:20:27.147059 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:20:27.147100 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:20:27.147458 (+ 358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:27.147918 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:27.148602 (+ 684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:27.148887 (+ 285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:27.148963 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:27.150000 (+ 1037us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:20:27.150002 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:20:27.150002 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:20:27.150009 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:20:27.151685 (+ 1676us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:20:27.152045 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:20:27.152046 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:20:27.152046 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:20:27.152113 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:20:27.152300 (+ 187us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:20:27.152301 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:20:27.152301 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:20:27.152496 (+ 195us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:20:27.152576 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0809 16:20:27.152870 (+ 294us) server_negotiation.cc:299] Negotiation successful
0809 16:20:27.152930 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260809 16:20:27.153102 24954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:27.146750 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:38211 (local address 127.21.51.130:56945)
0809 16:20:27.146928 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0809 16:20:27.146938 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 16:20:27.146998 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:20:27.147115 (+ 117us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:20:27.147116 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:20:27.147313 (+ 197us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:20:27.147371 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:20:27.147373 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:20:27.148014 (+ 641us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:20:27.148015 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:20:27.148519 (+ 504us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:20:27.148521 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:20:27.148962 (+ 441us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:20:27.148963 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:20:27.149044 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:27.149479 (+ 435us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:20:27.149484 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:20:27.149877 (+ 393us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:20:27.151796 (+ 1919us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:20:27.151797 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:20:27.151797 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:20:27.151970 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:20:27.152180 (+ 210us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:20:27.152180 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:20:27.152181 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:20:27.152235 (+ 54us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:20:27.152678 (+ 443us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:20:27.152679 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:20:27.152761 (+ 82us) client_negotiation.cc:769] Sending connection context
0809 16:20:27.152847 (+ 86us) client_negotiation.cc:240] Negotiation successful
0809 16:20:27.152965 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":91,"threads_started":1}
WARNING: no policy specified for kudu/127.21.51.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.51.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.51.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.131.keytab.
Entry for principal kudu/127.21.51.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.131.keytab.
I20260809 16:20:27.177659 21710 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrNqUCn/build/release/bin/kudu
/tmp/dist-test-taskrNqUCn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786292417722232-21710-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786292417722232-21710-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786292417722232-21710-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786292417722232-21710-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.51.131:0
--local_ip_for_outbound_sockets=127.21.51.131
--webserver_interface=127.21.51.131
--webserver_port=0
--tserver_master_addrs=127.21.51.190:38211
--keytab_file=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.131.keytab
--principal=kudu/127.21.51.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.51.148:36335
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786292417722232-21710-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786292417722232-21710-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786292417722232-21710-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:20:27.260969 24963 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:20:27.261159 24963 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:20:27.261184 24963 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:20:27.262341 24963 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:20:27.262423 24963 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:20:27.262446 24963 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260809 16:20:27.262458 24963 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 16:20:27.262475 24963 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.51.131
I20260809 16:20:27.264123 24963 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.51.148:36335
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786292417722232-21710-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786292417722232-21710-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.131.keytab
--principal=kudu/127.21.51.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.51.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786292417722232-21710-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.51.131
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.21.51.190:38211
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24963
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.51.131
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786292417722232-21710-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:20:27.264348 24963 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:20:27.264528 24963 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:20:27.269644 24970 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:20:27.269644 24973 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:20:27.269657 24971 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:20:27.269884 24963 server_base.cc:1034] running on GCE node
I20260809 16:20:27.270066 24963 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:20:27.270304 24963 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:20:27.271466 24963 hybrid_clock.cc:648] HybridClock initialized: now 1786292427271454 us; error 29 us; skew 500 ppm
Aug 09 16:20:27 dist-test-slave-s11t krb5kdc[24339](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292427, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.51.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:20:27.274022 24963 init.cc:377] Logged in from keytab as kudu/127.21.51.131@KRBTEST.COM (short username kudu)
I20260809 16:20:27.274447 24963 webserver.cc:460] Webserver started at http://127.21.51.131:34861/ using document root <none> and password file <none>
I20260809 16:20:27.274639 24963 fs_manager.cc:359] Metadata directory not provided
I20260809 16:20:27.274680 24963 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:20:27.274776 24963 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:20:27.275740 24963 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786292417722232-21710-0/minicluster-data/ts-2/data/instance:
uuid: "b40e1c9e9a4640108769f458b51d8690"
format_stamp: "Formatted at 2026-08-09 16:20:27 on dist-test-slave-s11t"
server_key: "1a2860e140f3307a53e768eea3d7df73"
server_key_iv: "67789cc3efd64cbd9eff99b83c4d7a69"
server_key_version: "encryptionkey@0"
I20260809 16:20:27.276044 24963 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786292417722232-21710-0/minicluster-data/ts-2/wal/instance:
uuid: "b40e1c9e9a4640108769f458b51d8690"
format_stamp: "Formatted at 2026-08-09 16:20:27 on dist-test-slave-s11t"
server_key: "1a2860e140f3307a53e768eea3d7df73"
server_key_iv: "67789cc3efd64cbd9eff99b83c4d7a69"
server_key_version: "encryptionkey@0"
I20260809 16:20:27.277383 24963 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 16:20:27.278121 24983 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:20:27.278358 24963 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:20:27.278430 24963 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786292417722232-21710-0/minicluster-data/ts-2/data,/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786292417722232-21710-0/minicluster-data/ts-2/wal
uuid: "b40e1c9e9a4640108769f458b51d8690"
format_stamp: "Formatted at 2026-08-09 16:20:27 on dist-test-slave-s11t"
server_key: "1a2860e140f3307a53e768eea3d7df73"
server_key_iv: "67789cc3efd64cbd9eff99b83c4d7a69"
server_key_version: "encryptionkey@0"
I20260809 16:20:27.278506 24963 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786292417722232-21710-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786292417722232-21710-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786292417722232-21710-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:20:27.288552 24963 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:20:27.290411 24963 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:20:27.290563 24963 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:20:27.291133 24963 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:20:27.291180 24963 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:20:27.291203 24963 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:20:27.291215 24963 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 16:20:27 dist-test-slave-s11t krb5kdc[24339](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292427, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.51.131@KRBTEST.COM for kudu/127.21.51.190@KRBTEST.COM
I20260809 16:20:27.298633 24963 rpc_server.cc:307] RPC server started. Bound to: 127.21.51.131:45353
I20260809 16:20:27.298802 25170 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.51.131:45353 every 8 connection(s)
I20260809 16:20:27.299064 24963 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786292417722232-21710-0/minicluster-data/ts-2/data/info.pb
I20260809 16:20:27.301440 24488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:27.291524 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.51.131:43913 (local address 127.21.51.190:38211)
0809 16:20:27.291589 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0809 16:20:27.291592 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 16:20:27.291807 (+ 215us) server_negotiation.cc:407] Connection header received
0809 16:20:27.292315 (+ 508us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:20:27.292316 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:20:27.292329 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:20:27.292403 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:20:27.293146 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:27.293723 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:27.294764 (+ 1041us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:27.294976 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:27.295054 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:27.298260 (+ 3206us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:20:27.298262 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:20:27.298262 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:20:27.298269 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:20:27.299961 (+ 1692us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:20:27.300417 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:20:27.300418 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:20:27.300419 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:20:27.300483 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:20:27.300769 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:20:27.300771 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:20:27.300771 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:20:27.300903 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:20:27.300984 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0809 16:20:27.301266 (+ 282us) server_negotiation.cc:299] Negotiation successful
0809 16:20:27.301352 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260809 16:20:27.301625 24994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:27.291522 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:38211 (local address 127.21.51.131:43913)
0809 16:20:27.291691 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0809 16:20:27.291725 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0809 16:20:27.292208 (+ 483us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:20:27.292471 (+ 263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:20:27.292475 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:20:27.292840 (+ 365us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:20:27.293045 (+ 205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:20:27.293050 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:20:27.293816 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:20:27.293817 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:20:27.294668 (+ 851us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:20:27.294671 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:20:27.295062 (+ 391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:20:27.295063 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:20:27.295199 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:27.295751 (+ 552us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:20:27.295762 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:20:27.298139 (+ 2377us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:20:27.300052 (+ 1913us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:20:27.300059 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:20:27.300061 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:20:27.300327 (+ 266us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:20:27.300554 (+ 227us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:20:27.300555 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:20:27.300555 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:20:27.300673 (+ 118us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:20:27.300974 (+ 301us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:20:27.300981 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:20:27.301123 (+ 142us) client_negotiation.cc:769] Sending connection context
0809 16:20:27.301226 (+ 103us) client_negotiation.cc:240] Negotiation successful
0809 16:20:27.301394 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":84,"threads_started":1}
I20260809 16:20:27.302119 25171 heartbeater.cc:347] Connected to a master server at 127.21.51.190:38211
I20260809 16:20:27.302208 25171 heartbeater.cc:464] Registering TS with master...
I20260809 16:20:27.302418 25171 heartbeater.cc:511] Master 127.21.51.190:38211 requested a full tablet report, sending...
I20260809 16:20:27.302809 24410 ts_manager.cc:194] Registered new tserver with Master: b40e1c9e9a4640108769f458b51d8690 (127.21.51.131:45353)
I20260809 16:20:27.303306 24410 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.51.131@KRBTEST.COM'} at 127.21.51.131:43913
I20260809 16:20:27.303948 21710 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrNqUCn/build/release/bin/kudu as pid 24963
I20260809 16:20:27.304041 21710 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786292417722232-21710-0/minicluster-data/ts-2/wal/instance
I20260809 16:20:27.304217 21710 external_mini_cluster.cc:1428] Setting key 30024acb6ad91a5079cd42c489fdf559
I20260809 16:20:27.305104 21710 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 16:20:27.309834 24488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:27.303305 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.51.131:42903 (local address 127.21.51.190:38211)
0809 16:20:27.303353 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0809 16:20:27.303355 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:20:27.303440 (+ 85us) server_negotiation.cc:407] Connection header received
0809 16:20:27.303500 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:20:27.303500 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:20:27.303511 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:20:27.303576 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:20:27.303952 (+ 376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:27.304677 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:27.305698 (+ 1021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:27.306063 (+ 365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:27.306147 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:27.307100 (+ 953us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:20:27.307101 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:20:27.307102 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:20:27.307108 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:20:27.308662 (+ 1554us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:20:27.309068 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:20:27.309069 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:20:27.309070 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:20:27.309127 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:20:27.309334 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:20:27.309334 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:20:27.309334 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:20:27.309431 (+ 97us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:20:27.309482 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0809 16:20:27.309643 (+ 161us) server_negotiation.cc:299] Negotiation successful
0809 16:20:27.309706 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260809 16:20:27.309906 25179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:27.303233 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:38211 (local address 127.21.51.131:42903)
0809 16:20:27.303356 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0809 16:20:27.303367 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 16:20:27.303449 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:20:27.303578 (+ 129us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:20:27.303578 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:20:27.303794 (+ 216us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:20:27.303863 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:20:27.303865 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:20:27.304794 (+ 929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:20:27.304795 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:20:27.305626 (+ 831us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:20:27.305629 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:20:27.306155 (+ 526us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:20:27.306156 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:20:27.306251 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:27.306716 (+ 465us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:20:27.306720 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:20:27.306993 (+ 273us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:20:27.308751 (+ 1758us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:20:27.308752 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:20:27.308753 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:20:27.308982 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:20:27.309195 (+ 213us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:20:27.309196 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:20:27.309197 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:20:27.309270 (+ 73us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:20:27.309494 (+ 224us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:20:27.309495 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:20:27.309544 (+ 49us) client_negotiation.cc:769] Sending connection context
0809 16:20:27.309616 (+ 72us) client_negotiation.cc:240] Negotiation successful
0809 16:20:27.309757 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":95,"thread_start_us":71,"threads_started":1}
Aug 09 16:20:27 dist-test-slave-s11t krb5kdc[24339](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292427, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 09 16:20:27 dist-test-slave-s11t krb5kdc[24339](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292427, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.51.190@KRBTEST.COM
I20260809 16:20:27.330339 24488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:27.323270 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60116 (local address 127.21.51.190:38211)
0809 16:20:27.323330 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0809 16:20:27.323332 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:20:27.323342 (+ 10us) server_negotiation.cc:407] Connection header received
0809 16:20:27.323390 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:20:27.323391 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:20:27.323407 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:20:27.323466 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:20:27.324047 (+ 581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:27.324524 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:27.325604 (+ 1080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:27.325959 (+ 355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:27.326045 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:27.328204 (+ 2159us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:20:27.328206 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:20:27.328206 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:20:27.328212 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:20:27.329454 (+ 1242us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:20:27.329759 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:20:27.329760 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:20:27.329761 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:20:27.329810 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:20:27.329973 (+ 163us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:20:27.329973 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:20:27.329973 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:20:27.330067 (+ 94us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:20:27.330142 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0809 16:20:27.330221 (+ 79us) server_negotiation.cc:299] Negotiation successful
0809 16:20:27.330277 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260809 16:20:27.332149 24410 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60116:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 16:20:27.333848 24410 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 16:20:27.340523 25198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:27.336380 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.131:45353 (local address 127.0.0.1:35402)
0809 16:20:27.336765 (+ 385us) negotiation.cc:107] Waiting for socket to connect
0809 16:20:27.336812 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0809 16:20:27.336926 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:20:27.337215 (+ 289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:20:27.337218 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:20:27.337221 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:20:27.337285 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:20:27.337286 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:20:27.338273 (+ 987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:20:27.338274 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:20:27.339256 (+ 982us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:20:27.339259 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:20:27.340231 (+ 972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:20:27.340232 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:20:27.340287 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:27.340295 (+ 8us) client_negotiation.cc:769] Sending connection context
0809 16:20:27.340374 (+ 79us) client_negotiation.cc:240] Negotiation successful
0809 16:20:27.340430 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":352,"spinlock_wait_cycles":768,"thread_start_us":93,"threads_started":1}
I20260809 16:20:27.340530 25196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:27.336480 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35402 (local address 127.21.51.131:45353)
0809 16:20:27.336636 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0809 16:20:27.336640 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 16:20:27.336879 (+ 239us) server_negotiation.cc:407] Connection header received
0809 16:20:27.337048 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:20:27.337051 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:20:27.337073 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:20:27.337154 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:20:27.337376 (+ 222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:27.338194 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:27.339359 (+ 1165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:27.340099 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:27.340141 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:27.340183 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0809 16:20:27.340399 (+ 216us) server_negotiation.cc:299] Negotiation successful
0809 16:20:27.340434 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":122,"thread_start_us":90,"threads_started":1}
I20260809 16:20:27.341061 25194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:27.336381 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.129:39087 (local address 127.0.0.1:44898)
0809 16:20:27.337101 (+ 720us) negotiation.cc:107] Waiting for socket to connect
0809 16:20:27.337111 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 16:20:27.337201 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:20:27.337456 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:20:27.337456 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:20:27.337458 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:20:27.337515 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:20:27.337517 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:20:27.338876 (+ 1359us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:20:27.338877 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:20:27.339835 (+ 958us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:20:27.339838 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:20:27.340797 (+ 959us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:20:27.340798 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:20:27.340861 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:27.340865 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 16:20:27.340940 (+ 75us) client_negotiation.cc:240] Negotiation successful
0809 16:20:27.340985 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":699,"mutex_wait_us":2,"thread_start_us":67,"threads_started":1}
I20260809 16:20:27.341109 25200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:27.336607 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44898 (local address 127.21.51.129:39087)
0809 16:20:27.336836 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0809 16:20:27.336840 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 16:20:27.337162 (+ 322us) server_negotiation.cc:407] Connection header received
0809 16:20:27.337287 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:20:27.337290 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:20:27.337310 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:20:27.337381 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:20:27.337592 (+ 211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:27.338769 (+ 1177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:27.339944 (+ 1175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:27.340719 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:27.340798 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:27.340850 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0809 16:20:27.340954 (+ 104us) server_negotiation.cc:299] Negotiation successful
0809 16:20:27.340995 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":191,"thread_start_us":156,"threads_started":1}
I20260809 16:20:27.341638 25193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:27.336593 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.130:43831 (local address 127.0.0.1:39228)
0809 16:20:27.337282 (+ 689us) negotiation.cc:107] Waiting for socket to connect
0809 16:20:27.337291 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 16:20:27.337337 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:20:27.337554 (+ 217us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:20:27.337555 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:20:27.337557 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:20:27.337596 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:20:27.337598 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:20:27.338850 (+ 1252us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:20:27.338851 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:20:27.339950 (+ 1099us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:20:27.339953 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:20:27.341395 (+ 1442us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:20:27.341396 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:20:27.341467 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:27.341471 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 16:20:27.341522 (+ 51us) client_negotiation.cc:240] Negotiation successful
0809 16:20:27.341568 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":677,"thread_start_us":106,"threads_started":1}
I20260809 16:20:27.341621 25060 tablet_service.cc:1467] Processing CreateTablet for tablet 999a50f911744834bd484eb981ce1348 (DEFAULT_TABLE table=test-table [id=93275e6ab7d0476bb1768c7e4c62771e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:20:27.341703 24613 tablet_service.cc:1467] Processing CreateTablet for tablet 999a50f911744834bd484eb981ce1348 (DEFAULT_TABLE table=test-table [id=93275e6ab7d0476bb1768c7e4c62771e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:20:27.342023 25202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:27.336852 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39228 (local address 127.21.51.130:43831)
0809 16:20:27.337020 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0809 16:20:27.337024 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 16:20:27.337400 (+ 376us) server_negotiation.cc:407] Connection header received
0809 16:20:27.337443 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:20:27.337446 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:20:27.337460 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:20:27.337540 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:20:27.337671 (+ 131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:27.338752 (+ 1081us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:27.340043 (+ 1291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:27.340993 (+ 950us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:27.341060 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:27.341866 (+ 806us) server_negotiation.cc:1035] Waiting for connection context
0809 16:20:27.341893 (+ 27us) server_negotiation.cc:299] Negotiation successful
0809 16:20:27.341932 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135,"thread_start_us":102,"threads_started":1}
I20260809 16:20:27.342026 25060 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 999a50f911744834bd484eb981ce1348. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:20:27.342026 24613 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 999a50f911744834bd484eb981ce1348. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:20:27.342679 24835 tablet_service.cc:1467] Processing CreateTablet for tablet 999a50f911744834bd484eb981ce1348 (DEFAULT_TABLE table=test-table [id=93275e6ab7d0476bb1768c7e4c62771e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:20:27.343020 24835 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 999a50f911744834bd484eb981ce1348. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:20:27.345506 25203 tablet_bootstrap.cc:492] T 999a50f911744834bd484eb981ce1348 P b40e1c9e9a4640108769f458b51d8690: Bootstrap starting.
I20260809 16:20:27.346004 25205 tablet_bootstrap.cc:492] T 999a50f911744834bd484eb981ce1348 P 4f1796cdf8d4421fb7f9d0f614743bdc: Bootstrap starting.
I20260809 16:20:27.346351 25206 tablet_bootstrap.cc:492] T 999a50f911744834bd484eb981ce1348 P 18fb7927ff43460790088c7b6465d2fb: Bootstrap starting.
I20260809 16:20:27.346637 25203 tablet_bootstrap.cc:654] T 999a50f911744834bd484eb981ce1348 P b40e1c9e9a4640108769f458b51d8690: Neither blocks nor log segments found. Creating new log.
I20260809 16:20:27.346661 25205 tablet_bootstrap.cc:654] T 999a50f911744834bd484eb981ce1348 P 4f1796cdf8d4421fb7f9d0f614743bdc: Neither blocks nor log segments found. Creating new log.
I20260809 16:20:27.347021 25203 log.cc:824] T 999a50f911744834bd484eb981ce1348 P b40e1c9e9a4640108769f458b51d8690: Log is configured to *not* fsync() on all Append() calls
I20260809 16:20:27.347036 25206 tablet_bootstrap.cc:654] T 999a50f911744834bd484eb981ce1348 P 18fb7927ff43460790088c7b6465d2fb: Neither blocks nor log segments found. Creating new log.
I20260809 16:20:27.347061 25205 log.cc:824] T 999a50f911744834bd484eb981ce1348 P 4f1796cdf8d4421fb7f9d0f614743bdc: Log is configured to *not* fsync() on all Append() calls
I20260809 16:20:27.347363 25206 log.cc:824] T 999a50f911744834bd484eb981ce1348 P 18fb7927ff43460790088c7b6465d2fb: Log is configured to *not* fsync() on all Append() calls
I20260809 16:20:27.347839 25203 tablet_bootstrap.cc:492] T 999a50f911744834bd484eb981ce1348 P b40e1c9e9a4640108769f458b51d8690: No bootstrap required, opened a new log
I20260809 16:20:27.347851 25205 tablet_bootstrap.cc:492] T 999a50f911744834bd484eb981ce1348 P 4f1796cdf8d4421fb7f9d0f614743bdc: No bootstrap required, opened a new log
I20260809 16:20:27.347918 25203 ts_tablet_manager.cc:1397] T 999a50f911744834bd484eb981ce1348 P b40e1c9e9a4640108769f458b51d8690: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 16:20:27.348040 25205 ts_tablet_manager.cc:1397] T 999a50f911744834bd484eb981ce1348 P 4f1796cdf8d4421fb7f9d0f614743bdc: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 16:20:27.348245 25206 tablet_bootstrap.cc:492] T 999a50f911744834bd484eb981ce1348 P 18fb7927ff43460790088c7b6465d2fb: No bootstrap required, opened a new log
I20260809 16:20:27.348320 25206 ts_tablet_manager.cc:1397] T 999a50f911744834bd484eb981ce1348 P 18fb7927ff43460790088c7b6465d2fb: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 16:20:27.349603 25205 raft_consensus.cc:348] T 999a50f911744834bd484eb981ce1348 P 4f1796cdf8d4421fb7f9d0f614743bdc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b40e1c9e9a4640108769f458b51d8690" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 45353 } } peers { permanent_uuid: "18fb7927ff43460790088c7b6465d2fb" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 43831 } } peers { permanent_uuid: "4f1796cdf8d4421fb7f9d0f614743bdc" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 39087 } }
I20260809 16:20:27.349756 25205 raft_consensus.cc:374] T 999a50f911744834bd484eb981ce1348 P 4f1796cdf8d4421fb7f9d0f614743bdc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:20:27.349800 25205 raft_consensus.cc:729] T 999a50f911744834bd484eb981ce1348 P 4f1796cdf8d4421fb7f9d0f614743bdc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f1796cdf8d4421fb7f9d0f614743bdc, State: Initialized, Role: FOLLOWER
I20260809 16:20:27.349922 25205 consensus_queue.cc:260] T 999a50f911744834bd484eb981ce1348 P 4f1796cdf8d4421fb7f9d0f614743bdc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b40e1c9e9a4640108769f458b51d8690" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 45353 } } peers { permanent_uuid: "18fb7927ff43460790088c7b6465d2fb" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 43831 } } peers { permanent_uuid: "4f1796cdf8d4421fb7f9d0f614743bdc" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 39087 } }
I20260809 16:20:27.349997 25203 raft_consensus.cc:348] T 999a50f911744834bd484eb981ce1348 P b40e1c9e9a4640108769f458b51d8690 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b40e1c9e9a4640108769f458b51d8690" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 45353 } } peers { permanent_uuid: "18fb7927ff43460790088c7b6465d2fb" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 43831 } } peers { permanent_uuid: "4f1796cdf8d4421fb7f9d0f614743bdc" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 39087 } }
I20260809 16:20:27.350164 25203 raft_consensus.cc:374] T 999a50f911744834bd484eb981ce1348 P b40e1c9e9a4640108769f458b51d8690 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:20:27.350211 25203 raft_consensus.cc:729] T 999a50f911744834bd484eb981ce1348 P b40e1c9e9a4640108769f458b51d8690 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b40e1c9e9a4640108769f458b51d8690, State: Initialized, Role: FOLLOWER
I20260809 16:20:27.350222 25205 ts_tablet_manager.cc:1428] T 999a50f911744834bd484eb981ce1348 P 4f1796cdf8d4421fb7f9d0f614743bdc: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260809 16:20:27.350368 24720 heartbeater.cc:503] Master 127.21.51.190:38211 was elected leader, sending a full tablet report...
I20260809 16:20:27.350327 25203 consensus_queue.cc:260] T 999a50f911744834bd484eb981ce1348 P b40e1c9e9a4640108769f458b51d8690 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b40e1c9e9a4640108769f458b51d8690" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 45353 } } peers { permanent_uuid: "18fb7927ff43460790088c7b6465d2fb" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 43831 } } peers { permanent_uuid: "4f1796cdf8d4421fb7f9d0f614743bdc" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 39087 } }
I20260809 16:20:27.350406 25206 raft_consensus.cc:348] T 999a50f911744834bd484eb981ce1348 P 18fb7927ff43460790088c7b6465d2fb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b40e1c9e9a4640108769f458b51d8690" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 45353 } } peers { permanent_uuid: "18fb7927ff43460790088c7b6465d2fb" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 43831 } } peers { permanent_uuid: "4f1796cdf8d4421fb7f9d0f614743bdc" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 39087 } }
I20260809 16:20:27.350566 25206 raft_consensus.cc:374] T 999a50f911744834bd484eb981ce1348 P 18fb7927ff43460790088c7b6465d2fb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:20:27.350597 25203 ts_tablet_manager.cc:1428] T 999a50f911744834bd484eb981ce1348 P b40e1c9e9a4640108769f458b51d8690: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 16:20:27.350613 25206 raft_consensus.cc:729] T 999a50f911744834bd484eb981ce1348 P 18fb7927ff43460790088c7b6465d2fb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 18fb7927ff43460790088c7b6465d2fb, State: Initialized, Role: FOLLOWER
I20260809 16:20:27.350747 25171 heartbeater.cc:503] Master 127.21.51.190:38211 was elected leader, sending a full tablet report...
I20260809 16:20:27.350739 25206 consensus_queue.cc:260] T 999a50f911744834bd484eb981ce1348 P 18fb7927ff43460790088c7b6465d2fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b40e1c9e9a4640108769f458b51d8690" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 45353 } } peers { permanent_uuid: "18fb7927ff43460790088c7b6465d2fb" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 43831 } } peers { permanent_uuid: "4f1796cdf8d4421fb7f9d0f614743bdc" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 39087 } }
I20260809 16:20:27.351030 25206 ts_tablet_manager.cc:1428] T 999a50f911744834bd484eb981ce1348 P 18fb7927ff43460790088c7b6465d2fb: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 16:20:27.351051 24943 heartbeater.cc:503] Master 127.21.51.190:38211 was elected leader, sending a full tablet report...
W20260809 16:20:27.394071 24944 tablet.cc:2369] T 999a50f911744834bd484eb981ce1348 P 18fb7927ff43460790088c7b6465d2fb: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 16:20:27.468824 24721 tablet.cc:2369] T 999a50f911744834bd484eb981ce1348 P 4f1796cdf8d4421fb7f9d0f614743bdc: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 16:20:27.537418 25214 raft_consensus.cc:482] T 999a50f911744834bd484eb981ce1348 P 18fb7927ff43460790088c7b6465d2fb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 16:20:27.537555 25214 raft_consensus.cc:504] T 999a50f911744834bd484eb981ce1348 P 18fb7927ff43460790088c7b6465d2fb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b40e1c9e9a4640108769f458b51d8690" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 45353 } } peers { permanent_uuid: "18fb7927ff43460790088c7b6465d2fb" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 43831 } } peers { permanent_uuid: "4f1796cdf8d4421fb7f9d0f614743bdc" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 39087 } }
I20260809 16:20:27.537919 25214 leader_election.cc:288] T 999a50f911744834bd484eb981ce1348 P 18fb7927ff43460790088c7b6465d2fb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b40e1c9e9a4640108769f458b51d8690 (127.21.51.131:45353), 4f1796cdf8d4421fb7f9d0f614743bdc (127.21.51.129:39087)
I20260809 16:20:27.541154 25196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:27.538226 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.51.130:45603 (local address 127.21.51.131:45353)
0809 16:20:27.538275 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0809 16:20:27.538277 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:20:27.538313 (+ 36us) server_negotiation.cc:407] Connection header received
0809 16:20:27.538365 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:20:27.538366 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:20:27.538398 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:20:27.538458 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:20:27.538674 (+ 216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:27.539215 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:27.540368 (+ 1153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:27.540792 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:27.540874 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:27.540882 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0809 16:20:27.541051 (+ 169us) server_negotiation.cc:299] Negotiation successful
0809 16:20:27.541087 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260809 16:20:27.541157 24768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:27.538158 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.131:45353 (local address 127.21.51.130:45603)
0809 16:20:27.538245 (+ 87us) negotiation.cc:107] Waiting for socket to connect
0809 16:20:27.538259 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 16:20:27.538303 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:20:27.538472 (+ 169us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:20:27.538473 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:20:27.538474 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:20:27.538571 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:20:27.538573 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:20:27.539317 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:20:27.539318 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:20:27.540249 (+ 931us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:20:27.540252 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:20:27.540872 (+ 620us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:20:27.540873 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:20:27.540963 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:27.540966 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 16:20:27.541030 (+ 64us) client_negotiation.cc:240] Negotiation successful
0809 16:20:27.541075 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":65}
I20260809 16:20:27.541157 25200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:27.538245 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.51.130:38059 (local address 127.21.51.129:39087)
0809 16:20:27.538315 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0809 16:20:27.538317 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:20:27.538547 (+ 230us) server_negotiation.cc:407] Connection header received
0809 16:20:27.538582 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:20:27.538583 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:20:27.538593 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:20:27.538652 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:20:27.538824 (+ 172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:27.539332 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:27.540337 (+ 1005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:27.540786 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:27.540870 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:27.540881 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0809 16:20:27.541042 (+ 161us) server_negotiation.cc:299] Negotiation successful
0809 16:20:27.541090 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260809 16:20:27.541170 25215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:27.538166 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.129:39087 (local address 127.21.51.130:38059)
0809 16:20:27.538457 (+ 291us) negotiation.cc:107] Waiting for socket to connect
0809 16:20:27.538468 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 16:20:27.538529 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:20:27.538671 (+ 142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:20:27.538672 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:20:27.538673 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:20:27.538747 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:20:27.538749 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:20:27.539423 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:20:27.539424 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:20:27.540219 (+ 795us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:20:27.540222 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:20:27.540881 (+ 659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:20:27.540883 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:20:27.540964 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:27.540967 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 16:20:27.541034 (+ 67us) client_negotiation.cc:240] Negotiation successful
0809 16:20:27.541076 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":271,"mutex_wait_us":69,"thread_start_us":147,"threads_started":1}
I20260809 16:20:27.541544 24647 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "999a50f911744834bd484eb981ce1348" candidate_uuid: "18fb7927ff43460790088c7b6465d2fb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f1796cdf8d4421fb7f9d0f614743bdc" is_pre_election: true
I20260809 16:20:27.541575 25091 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "999a50f911744834bd484eb981ce1348" candidate_uuid: "18fb7927ff43460790088c7b6465d2fb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b40e1c9e9a4640108769f458b51d8690" is_pre_election: true
I20260809 16:20:27.541733 25091 raft_consensus.cc:2454] T 999a50f911744834bd484eb981ce1348 P b40e1c9e9a4640108769f458b51d8690 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 18fb7927ff43460790088c7b6465d2fb in term 0.
I20260809 16:20:27.541751 24647 raft_consensus.cc:2454] T 999a50f911744834bd484eb981ce1348 P 4f1796cdf8d4421fb7f9d0f614743bdc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 18fb7927ff43460790088c7b6465d2fb in term 0.
I20260809 16:20:27.542001 24761 leader_election.cc:302] T 999a50f911744834bd484eb981ce1348 P 18fb7927ff43460790088c7b6465d2fb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 18fb7927ff43460790088c7b6465d2fb, 4f1796cdf8d4421fb7f9d0f614743bdc; no voters:
I20260809 16:20:27.542138 25214 raft_consensus.cc:2781] T 999a50f911744834bd484eb981ce1348 P 18fb7927ff43460790088c7b6465d2fb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 16:20:27.542202 25214 raft_consensus.cc:482] T 999a50f911744834bd484eb981ce1348 P 18fb7927ff43460790088c7b6465d2fb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 16:20:27.542225 25214 raft_consensus.cc:3037] T 999a50f911744834bd484eb981ce1348 P 18fb7927ff43460790088c7b6465d2fb [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:20:27.543155 25214 raft_consensus.cc:504] T 999a50f911744834bd484eb981ce1348 P 18fb7927ff43460790088c7b6465d2fb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b40e1c9e9a4640108769f458b51d8690" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 45353 } } peers { permanent_uuid: "18fb7927ff43460790088c7b6465d2fb" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 43831 } } peers { permanent_uuid: "4f1796cdf8d4421fb7f9d0f614743bdc" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 39087 } }
I20260809 16:20:27.543298 25214 leader_election.cc:288] T 999a50f911744834bd484eb981ce1348 P 18fb7927ff43460790088c7b6465d2fb [CANDIDATE]: Term 1 election: Requested vote from peers b40e1c9e9a4640108769f458b51d8690 (127.21.51.131:45353), 4f1796cdf8d4421fb7f9d0f614743bdc (127.21.51.129:39087)
I20260809 16:20:27.543488 25091 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "999a50f911744834bd484eb981ce1348" candidate_uuid: "18fb7927ff43460790088c7b6465d2fb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b40e1c9e9a4640108769f458b51d8690"
I20260809 16:20:27.543483 24647 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "999a50f911744834bd484eb981ce1348" candidate_uuid: "18fb7927ff43460790088c7b6465d2fb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f1796cdf8d4421fb7f9d0f614743bdc"
I20260809 16:20:27.543572 25091 raft_consensus.cc:3037] T 999a50f911744834bd484eb981ce1348 P b40e1c9e9a4640108769f458b51d8690 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:20:27.543572 24647 raft_consensus.cc:3037] T 999a50f911744834bd484eb981ce1348 P 4f1796cdf8d4421fb7f9d0f614743bdc [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:20:27.544453 25091 raft_consensus.cc:2454] T 999a50f911744834bd484eb981ce1348 P b40e1c9e9a4640108769f458b51d8690 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 18fb7927ff43460790088c7b6465d2fb in term 1.
I20260809 16:20:27.544458 24647 raft_consensus.cc:2454] T 999a50f911744834bd484eb981ce1348 P 4f1796cdf8d4421fb7f9d0f614743bdc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 18fb7927ff43460790088c7b6465d2fb in term 1.
I20260809 16:20:27.544675 24762 leader_election.cc:302] T 999a50f911744834bd484eb981ce1348 P 18fb7927ff43460790088c7b6465d2fb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 18fb7927ff43460790088c7b6465d2fb, b40e1c9e9a4640108769f458b51d8690; no voters:
I20260809 16:20:27.544796 25214 raft_consensus.cc:2781] T 999a50f911744834bd484eb981ce1348 P 18fb7927ff43460790088c7b6465d2fb [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:20:27.544929 25214 raft_consensus.cc:686] T 999a50f911744834bd484eb981ce1348 P 18fb7927ff43460790088c7b6465d2fb [term 1 LEADER]: Becoming Leader. State: Replica: 18fb7927ff43460790088c7b6465d2fb, State: Running, Role: LEADER
I20260809 16:20:27.545024 25214 consensus_queue.cc:237] T 999a50f911744834bd484eb981ce1348 P 18fb7927ff43460790088c7b6465d2fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b40e1c9e9a4640108769f458b51d8690" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 45353 } } peers { permanent_uuid: "18fb7927ff43460790088c7b6465d2fb" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 43831 } } peers { permanent_uuid: "4f1796cdf8d4421fb7f9d0f614743bdc" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 39087 } }
I20260809 16:20:27.545914 24411 catalog_manager.cc:5495] T 999a50f911744834bd484eb981ce1348 P 18fb7927ff43460790088c7b6465d2fb reported cstate change: term changed from 0 to 1, leader changed from <none> to 18fb7927ff43460790088c7b6465d2fb (127.21.51.130). New cstate: current_term: 1 leader_uuid: "18fb7927ff43460790088c7b6465d2fb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b40e1c9e9a4640108769f458b51d8690" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 45353 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "18fb7927ff43460790088c7b6465d2fb" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 43831 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4f1796cdf8d4421fb7f9d0f614743bdc" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 39087 } health_report { overall_health: UNKNOWN } } }
W20260809 16:20:27.550351 25172 tablet.cc:2369] T 999a50f911744834bd484eb981ce1348 P b40e1c9e9a4640108769f458b51d8690: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 09 16:20:27 dist-test-slave-s11t krb5kdc[24339](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292426, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.51.190@KRBTEST.COM for kudu/127.21.51.190@KRBTEST.COM
I20260809 16:20:27.588840 25226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:27.581546 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:38211 (local address 127.0.0.1:60124)
0809 16:20:27.581740 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0809 16:20:27.581752 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 16:20:27.581845 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:20:27.581996 (+ 151us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:20:27.581997 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:20:27.582356 (+ 359us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:20:27.582406 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:20:27.582409 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:20:27.583118 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:20:27.583119 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:20:27.583659 (+ 540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:20:27.583661 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:20:27.584017 (+ 356us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:20:27.584017 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:20:27.584060 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:27.584583 (+ 523us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:20:27.584588 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:20:27.586128 (+ 1540us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:20:27.587630 (+ 1502us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:20:27.587637 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:20:27.587639 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:20:27.587850 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:20:27.588068 (+ 218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:20:27.588068 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:20:27.588069 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:20:27.588132 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:20:27.588479 (+ 347us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:20:27.588486 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:20:27.588532 (+ 46us) client_negotiation.cc:769] Sending connection context
0809 16:20:27.588593 (+ 61us) client_negotiation.cc:240] Negotiation successful
0809 16:20:27.588726 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":108,"threads_started":1}
I20260809 16:20:27.588960 24488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:27.581639 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60124 (local address 127.21.51.190:38211)
0809 16:20:27.581705 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0809 16:20:27.581707 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:20:27.581826 (+ 119us) server_negotiation.cc:407] Connection header received
0809 16:20:27.581911 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:20:27.581912 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:20:27.581922 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:20:27.581986 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:20:27.582473 (+ 487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:27.583042 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:27.583735 (+ 693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:27.583948 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:27.584004 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:27.586222 (+ 2218us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:20:27.586224 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:20:27.586224 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:20:27.586233 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:20:27.587547 (+ 1314us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:20:27.587919 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:20:27.587920 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:20:27.587920 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:20:27.587988 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:20:27.588222 (+ 234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:20:27.588223 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:20:27.588223 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:20:27.588394 (+ 171us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:20:27.588459 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0809 16:20:27.588661 (+ 202us) server_negotiation.cc:299] Negotiation successful
0809 16:20:27.588832 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260809 16:20:27.591100 24411 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.21.51.190@KRBTEST.COM'} at 127.0.0.1:60124:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260809 16:20:27.591349 24411 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 16:20:27.592933 24613 tablet_service.cc:1467] Processing CreateTablet for tablet 375a3271b4e2497995b936b9417f0d7a (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=e91818ea4ae8419082e5d4a582744960]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 16:20:27.592991 24835 tablet_service.cc:1467] Processing CreateTablet for tablet 375a3271b4e2497995b936b9417f0d7a (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=e91818ea4ae8419082e5d4a582744960]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 16:20:27.593006 25060 tablet_service.cc:1467] Processing CreateTablet for tablet 375a3271b4e2497995b936b9417f0d7a (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=e91818ea4ae8419082e5d4a582744960]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 16:20:27.593140 24835 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 375a3271b4e2497995b936b9417f0d7a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:20:27.593140 24613 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 375a3271b4e2497995b936b9417f0d7a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:20:27.593210 25060 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 375a3271b4e2497995b936b9417f0d7a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:20:27.594456 25203 tablet_bootstrap.cc:492] T 375a3271b4e2497995b936b9417f0d7a P b40e1c9e9a4640108769f458b51d8690: Bootstrap starting.
I20260809 16:20:27.594774 25205 tablet_bootstrap.cc:492] T 375a3271b4e2497995b936b9417f0d7a P 4f1796cdf8d4421fb7f9d0f614743bdc: Bootstrap starting.
I20260809 16:20:27.594775 25206 tablet_bootstrap.cc:492] T 375a3271b4e2497995b936b9417f0d7a P 18fb7927ff43460790088c7b6465d2fb: Bootstrap starting.
I20260809 16:20:27.595094 25203 tablet_bootstrap.cc:654] T 375a3271b4e2497995b936b9417f0d7a P b40e1c9e9a4640108769f458b51d8690: Neither blocks nor log segments found. Creating new log.
I20260809 16:20:27.595115 25205 tablet_bootstrap.cc:654] T 375a3271b4e2497995b936b9417f0d7a P 4f1796cdf8d4421fb7f9d0f614743bdc: Neither blocks nor log segments found. Creating new log.
I20260809 16:20:27.595227 25206 tablet_bootstrap.cc:654] T 375a3271b4e2497995b936b9417f0d7a P 18fb7927ff43460790088c7b6465d2fb: Neither blocks nor log segments found. Creating new log.
I20260809 16:20:27.595801 25205 tablet_bootstrap.cc:492] T 375a3271b4e2497995b936b9417f0d7a P 4f1796cdf8d4421fb7f9d0f614743bdc: No bootstrap required, opened a new log
I20260809 16:20:27.595858 25205 ts_tablet_manager.cc:1397] T 375a3271b4e2497995b936b9417f0d7a P 4f1796cdf8d4421fb7f9d0f614743bdc: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260809 16:20:27.595939 25206 tablet_bootstrap.cc:492] T 375a3271b4e2497995b936b9417f0d7a P 18fb7927ff43460790088c7b6465d2fb: No bootstrap required, opened a new log
I20260809 16:20:27.596017 25206 ts_tablet_manager.cc:1397] T 375a3271b4e2497995b936b9417f0d7a P 18fb7927ff43460790088c7b6465d2fb: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260809 16:20:27.596009 25205 raft_consensus.cc:348] T 375a3271b4e2497995b936b9417f0d7a P 4f1796cdf8d4421fb7f9d0f614743bdc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1796cdf8d4421fb7f9d0f614743bdc" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 39087 } } peers { permanent_uuid: "18fb7927ff43460790088c7b6465d2fb" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 43831 } } peers { permanent_uuid: "b40e1c9e9a4640108769f458b51d8690" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 45353 } }
I20260809 16:20:27.596103 25205 raft_consensus.cc:374] T 375a3271b4e2497995b936b9417f0d7a P 4f1796cdf8d4421fb7f9d0f614743bdc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:20:27.596130 25205 raft_consensus.cc:729] T 375a3271b4e2497995b936b9417f0d7a P 4f1796cdf8d4421fb7f9d0f614743bdc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f1796cdf8d4421fb7f9d0f614743bdc, State: Initialized, Role: FOLLOWER
I20260809 16:20:27.596168 25206 raft_consensus.cc:348] T 375a3271b4e2497995b936b9417f0d7a P 18fb7927ff43460790088c7b6465d2fb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1796cdf8d4421fb7f9d0f614743bdc" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 39087 } } peers { permanent_uuid: "18fb7927ff43460790088c7b6465d2fb" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 43831 } } peers { permanent_uuid: "b40e1c9e9a4640108769f458b51d8690" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 45353 } }
I20260809 16:20:27.596186 25205 consensus_queue.cc:260] T 375a3271b4e2497995b936b9417f0d7a P 4f1796cdf8d4421fb7f9d0f614743bdc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1796cdf8d4421fb7f9d0f614743bdc" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 39087 } } peers { permanent_uuid: "18fb7927ff43460790088c7b6465d2fb" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 43831 } } peers { permanent_uuid: "b40e1c9e9a4640108769f458b51d8690" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 45353 } }
I20260809 16:20:27.596241 25206 raft_consensus.cc:374] T 375a3271b4e2497995b936b9417f0d7a P 18fb7927ff43460790088c7b6465d2fb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:20:27.596274 25206 raft_consensus.cc:729] T 375a3271b4e2497995b936b9417f0d7a P 18fb7927ff43460790088c7b6465d2fb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 18fb7927ff43460790088c7b6465d2fb, State: Initialized, Role: FOLLOWER
I20260809 16:20:27.596338 25205 ts_tablet_manager.cc:1428] T 375a3271b4e2497995b936b9417f0d7a P 4f1796cdf8d4421fb7f9d0f614743bdc: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260809 16:20:27.596341 25206 consensus_queue.cc:260] T 375a3271b4e2497995b936b9417f0d7a P 18fb7927ff43460790088c7b6465d2fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1796cdf8d4421fb7f9d0f614743bdc" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 39087 } } peers { permanent_uuid: "18fb7927ff43460790088c7b6465d2fb" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 43831 } } peers { permanent_uuid: "b40e1c9e9a4640108769f458b51d8690" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 45353 } }
I20260809 16:20:27.596436 25206 ts_tablet_manager.cc:1428] T 375a3271b4e2497995b936b9417f0d7a P 18fb7927ff43460790088c7b6465d2fb: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260809 16:20:27.596400 25210 tablet_replica.cc:439] T 375a3271b4e2497995b936b9417f0d7a P 4f1796cdf8d4421fb7f9d0f614743bdc: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1796cdf8d4421fb7f9d0f614743bdc" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 39087 } } peers { permanent_uuid: "18fb7927ff43460790088c7b6465d2fb" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 43831 } } peers { permanent_uuid: "b40e1c9e9a4640108769f458b51d8690" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 45353 } } }
I20260809 16:20:27.596488 25210 tablet_replica.cc:442] T 375a3271b4e2497995b936b9417f0d7a P 4f1796cdf8d4421fb7f9d0f614743bdc: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 16:20:27.596575 25214 tablet_replica.cc:439] T 375a3271b4e2497995b936b9417f0d7a P 18fb7927ff43460790088c7b6465d2fb: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1796cdf8d4421fb7f9d0f614743bdc" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 39087 } } peers { permanent_uuid: "18fb7927ff43460790088c7b6465d2fb" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 43831 } } peers { permanent_uuid: "b40e1c9e9a4640108769f458b51d8690" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 45353 } } }
I20260809 16:20:27.596648 25214 tablet_replica.cc:442] T 375a3271b4e2497995b936b9417f0d7a P 18fb7927ff43460790088c7b6465d2fb: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 16:20:27.596783 25203 tablet_bootstrap.cc:492] T 375a3271b4e2497995b936b9417f0d7a P b40e1c9e9a4640108769f458b51d8690: No bootstrap required, opened a new log
I20260809 16:20:27.596838 25203 ts_tablet_manager.cc:1397] T 375a3271b4e2497995b936b9417f0d7a P b40e1c9e9a4640108769f458b51d8690: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 16:20:27.597002 25203 raft_consensus.cc:348] T 375a3271b4e2497995b936b9417f0d7a P b40e1c9e9a4640108769f458b51d8690 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1796cdf8d4421fb7f9d0f614743bdc" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 39087 } } peers { permanent_uuid: "18fb7927ff43460790088c7b6465d2fb" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 43831 } } peers { permanent_uuid: "b40e1c9e9a4640108769f458b51d8690" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 45353 } }
I20260809 16:20:27.597077 25203 raft_consensus.cc:374] T 375a3271b4e2497995b936b9417f0d7a P b40e1c9e9a4640108769f458b51d8690 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:20:27.597100 25203 raft_consensus.cc:729] T 375a3271b4e2497995b936b9417f0d7a P b40e1c9e9a4640108769f458b51d8690 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b40e1c9e9a4640108769f458b51d8690, State: Initialized, Role: FOLLOWER
I20260809 16:20:27.597157 25203 consensus_queue.cc:260] T 375a3271b4e2497995b936b9417f0d7a P b40e1c9e9a4640108769f458b51d8690 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1796cdf8d4421fb7f9d0f614743bdc" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 39087 } } peers { permanent_uuid: "18fb7927ff43460790088c7b6465d2fb" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 43831 } } peers { permanent_uuid: "b40e1c9e9a4640108769f458b51d8690" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 45353 } }
I20260809 16:20:27.597278 25203 ts_tablet_manager.cc:1428] T 375a3271b4e2497995b936b9417f0d7a P b40e1c9e9a4640108769f458b51d8690: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260809 16:20:27.597446 25213 tablet_replica.cc:439] T 375a3271b4e2497995b936b9417f0d7a P b40e1c9e9a4640108769f458b51d8690: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1796cdf8d4421fb7f9d0f614743bdc" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 39087 } } peers { permanent_uuid: "18fb7927ff43460790088c7b6465d2fb" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 43831 } } peers { permanent_uuid: "b40e1c9e9a4640108769f458b51d8690" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 45353 } } }
I20260809 16:20:27.597559 25213 tablet_replica.cc:442] T 375a3271b4e2497995b936b9417f0d7a P b40e1c9e9a4640108769f458b51d8690: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 16:20:27.661914 25214 raft_consensus.cc:482] T 375a3271b4e2497995b936b9417f0d7a P 18fb7927ff43460790088c7b6465d2fb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 16:20:27.662015 25214 raft_consensus.cc:504] T 375a3271b4e2497995b936b9417f0d7a P 18fb7927ff43460790088c7b6465d2fb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1796cdf8d4421fb7f9d0f614743bdc" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 39087 } } peers { permanent_uuid: "18fb7927ff43460790088c7b6465d2fb" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 43831 } } peers { permanent_uuid: "b40e1c9e9a4640108769f458b51d8690" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 45353 } }
I20260809 16:20:27.662173 25214 leader_election.cc:288] T 375a3271b4e2497995b936b9417f0d7a P 18fb7927ff43460790088c7b6465d2fb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4f1796cdf8d4421fb7f9d0f614743bdc (127.21.51.129:39087), b40e1c9e9a4640108769f458b51d8690 (127.21.51.131:45353)
I20260809 16:20:27.662427 24647 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "375a3271b4e2497995b936b9417f0d7a" candidate_uuid: "18fb7927ff43460790088c7b6465d2fb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f1796cdf8d4421fb7f9d0f614743bdc" is_pre_election: true
I20260809 16:20:27.662529 24647 raft_consensus.cc:2454] T 375a3271b4e2497995b936b9417f0d7a P 4f1796cdf8d4421fb7f9d0f614743bdc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 18fb7927ff43460790088c7b6465d2fb in term 0.
I20260809 16:20:27.662470 25091 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "375a3271b4e2497995b936b9417f0d7a" candidate_uuid: "18fb7927ff43460790088c7b6465d2fb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b40e1c9e9a4640108769f458b51d8690" is_pre_election: true
I20260809 16:20:27.662618 25091 raft_consensus.cc:2454] T 375a3271b4e2497995b936b9417f0d7a P b40e1c9e9a4640108769f458b51d8690 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 18fb7927ff43460790088c7b6465d2fb in term 0.
I20260809 16:20:27.662701 24761 leader_election.cc:302] T 375a3271b4e2497995b936b9417f0d7a P 18fb7927ff43460790088c7b6465d2fb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 18fb7927ff43460790088c7b6465d2fb, 4f1796cdf8d4421fb7f9d0f614743bdc; no voters:
I20260809 16:20:27.662837 25214 raft_consensus.cc:2781] T 375a3271b4e2497995b936b9417f0d7a P 18fb7927ff43460790088c7b6465d2fb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 16:20:27.662878 25214 raft_consensus.cc:482] T 375a3271b4e2497995b936b9417f0d7a P 18fb7927ff43460790088c7b6465d2fb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 16:20:27.662907 25214 raft_consensus.cc:3037] T 375a3271b4e2497995b936b9417f0d7a P 18fb7927ff43460790088c7b6465d2fb [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:20:27.663825 25214 raft_consensus.cc:504] T 375a3271b4e2497995b936b9417f0d7a P 18fb7927ff43460790088c7b6465d2fb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1796cdf8d4421fb7f9d0f614743bdc" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 39087 } } peers { permanent_uuid: "18fb7927ff43460790088c7b6465d2fb" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 43831 } } peers { permanent_uuid: "b40e1c9e9a4640108769f458b51d8690" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 45353 } }
I20260809 16:20:27.663970 25214 leader_election.cc:288] T 375a3271b4e2497995b936b9417f0d7a P 18fb7927ff43460790088c7b6465d2fb [CANDIDATE]: Term 1 election: Requested vote from peers 4f1796cdf8d4421fb7f9d0f614743bdc (127.21.51.129:39087), b40e1c9e9a4640108769f458b51d8690 (127.21.51.131:45353)
I20260809 16:20:27.664108 24647 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "375a3271b4e2497995b936b9417f0d7a" candidate_uuid: "18fb7927ff43460790088c7b6465d2fb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f1796cdf8d4421fb7f9d0f614743bdc"
I20260809 16:20:27.664244 24647 raft_consensus.cc:3037] T 375a3271b4e2497995b936b9417f0d7a P 4f1796cdf8d4421fb7f9d0f614743bdc [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:20:27.664242 25091 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "375a3271b4e2497995b936b9417f0d7a" candidate_uuid: "18fb7927ff43460790088c7b6465d2fb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b40e1c9e9a4640108769f458b51d8690"
I20260809 16:20:27.664315 25091 raft_consensus.cc:3037] T 375a3271b4e2497995b936b9417f0d7a P b40e1c9e9a4640108769f458b51d8690 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:20:27.665004 24647 raft_consensus.cc:2454] T 375a3271b4e2497995b936b9417f0d7a P 4f1796cdf8d4421fb7f9d0f614743bdc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 18fb7927ff43460790088c7b6465d2fb in term 1.
I20260809 16:20:27.665130 25091 raft_consensus.cc:2454] T 375a3271b4e2497995b936b9417f0d7a P b40e1c9e9a4640108769f458b51d8690 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 18fb7927ff43460790088c7b6465d2fb in term 1.
I20260809 16:20:27.665210 24761 leader_election.cc:302] T 375a3271b4e2497995b936b9417f0d7a P 18fb7927ff43460790088c7b6465d2fb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 18fb7927ff43460790088c7b6465d2fb, 4f1796cdf8d4421fb7f9d0f614743bdc; no voters:
I20260809 16:20:27.665328 25214 raft_consensus.cc:2781] T 375a3271b4e2497995b936b9417f0d7a P 18fb7927ff43460790088c7b6465d2fb [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:20:27.665401 25214 raft_consensus.cc:686] T 375a3271b4e2497995b936b9417f0d7a P 18fb7927ff43460790088c7b6465d2fb [term 1 LEADER]: Becoming Leader. State: Replica: 18fb7927ff43460790088c7b6465d2fb, State: Running, Role: LEADER
I20260809 16:20:27.665457 25214 consensus_queue.cc:237] T 375a3271b4e2497995b936b9417f0d7a P 18fb7927ff43460790088c7b6465d2fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1796cdf8d4421fb7f9d0f614743bdc" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 39087 } } peers { permanent_uuid: "18fb7927ff43460790088c7b6465d2fb" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 43831 } } peers { permanent_uuid: "b40e1c9e9a4640108769f458b51d8690" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 45353 } }
I20260809 16:20:27.665760 25216 tablet_replica.cc:439] T 375a3271b4e2497995b936b9417f0d7a P 18fb7927ff43460790088c7b6465d2fb: TxnStatusTablet state changed. Reason: New leader 18fb7927ff43460790088c7b6465d2fb. Latest consensus state: current_term: 1 leader_uuid: "18fb7927ff43460790088c7b6465d2fb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1796cdf8d4421fb7f9d0f614743bdc" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 39087 } } peers { permanent_uuid: "18fb7927ff43460790088c7b6465d2fb" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 43831 } } peers { permanent_uuid: "b40e1c9e9a4640108769f458b51d8690" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 45353 } } }
I20260809 16:20:27.665877 25216 tablet_replica.cc:442] T 375a3271b4e2497995b936b9417f0d7a P 18fb7927ff43460790088c7b6465d2fb: This TxnStatusTablet replica's current role is: LEADER
I20260809 16:20:27.666072 25233 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260809 16:20:27.666060 24411 catalog_manager.cc:5495] T 375a3271b4e2497995b936b9417f0d7a P 18fb7927ff43460790088c7b6465d2fb reported cstate change: term changed from 0 to 1, leader changed from <none> to 18fb7927ff43460790088c7b6465d2fb (127.21.51.130). New cstate: current_term: 1 leader_uuid: "18fb7927ff43460790088c7b6465d2fb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1796cdf8d4421fb7f9d0f614743bdc" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 39087 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "18fb7927ff43460790088c7b6465d2fb" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 43831 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b40e1c9e9a4640108769f458b51d8690" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 45353 } health_report { overall_health: UNKNOWN } } }
I20260809 16:20:27.666213 25233 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260809 16:20:27.674000 25226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:27.670575 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.130:43831 (local address 127.0.0.1:39242)
0809 16:20:27.670680 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0809 16:20:27.670693 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 16:20:27.670751 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:20:27.670952 (+ 201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:20:27.670952 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:20:27.670953 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 16:20:27.671017 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:20:27.671018 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:20:27.672074 (+ 1056us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:20:27.672075 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:20:27.672940 (+ 865us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:20:27.672943 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:20:27.673479 (+ 536us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:20:27.673480 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:20:27.673541 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:27.673556 (+ 15us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 16:20:27.673867 (+ 311us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 16:20:27.673874 (+ 7us) client_negotiation.cc:769] Sending connection context
0809 16:20:27.673899 (+ 25us) client_negotiation.cc:240] Negotiation successful
0809 16:20:27.673943 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":58}
I20260809 16:20:27.674041 25202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:27.670694 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39242 (local address 127.21.51.130:43831)
0809 16:20:27.670773 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0809 16:20:27.670776 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 16:20:27.670815 (+ 39us) server_negotiation.cc:407] Connection header received
0809 16:20:27.670848 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:20:27.670849 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:20:27.670868 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:20:27.670942 (+ 74us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 16:20:27.671085 (+ 143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:27.671956 (+ 871us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:27.673028 (+ 1072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:27.673392 (+ 364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:27.673478 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:27.673657 (+ 179us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 16:20:27.673792 (+ 135us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 16:20:27.673885 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0809 16:20:27.673924 (+ 39us) server_negotiation.cc:299] Negotiation successful
0809 16:20:27.673950 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260809 16:20:27.676177 24647 raft_consensus.cc:1261] T 375a3271b4e2497995b936b9417f0d7a P 4f1796cdf8d4421fb7f9d0f614743bdc [term 1 FOLLOWER]: Refusing update from remote peer 18fb7927ff43460790088c7b6465d2fb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 16:20:27.676189 25091 raft_consensus.cc:1261] T 375a3271b4e2497995b936b9417f0d7a P b40e1c9e9a4640108769f458b51d8690 [term 1 FOLLOWER]: Refusing update from remote peer 18fb7927ff43460790088c7b6465d2fb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 16:20:27.676561 25214 consensus_queue.cc:1035] T 375a3271b4e2497995b936b9417f0d7a P 18fb7927ff43460790088c7b6465d2fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "b40e1c9e9a4640108769f458b51d8690" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 45353 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 16:20:27.676740 25216 consensus_queue.cc:1035] T 375a3271b4e2497995b936b9417f0d7a P 18fb7927ff43460790088c7b6465d2fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1796cdf8d4421fb7f9d0f614743bdc" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 39087 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 16:20:27.678031 25213 tablet_replica.cc:439] T 375a3271b4e2497995b936b9417f0d7a P b40e1c9e9a4640108769f458b51d8690: TxnStatusTablet state changed. Reason: New leader 18fb7927ff43460790088c7b6465d2fb. Latest consensus state: current_term: 1 leader_uuid: "18fb7927ff43460790088c7b6465d2fb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1796cdf8d4421fb7f9d0f614743bdc" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 39087 } } peers { permanent_uuid: "18fb7927ff43460790088c7b6465d2fb" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 43831 } } peers { permanent_uuid: "b40e1c9e9a4640108769f458b51d8690" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 45353 } } }
I20260809 16:20:27.678148 25213 tablet_replica.cc:442] T 375a3271b4e2497995b936b9417f0d7a P b40e1c9e9a4640108769f458b51d8690: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 16:20:27.679095 25210 tablet_replica.cc:439] T 375a3271b4e2497995b936b9417f0d7a P 4f1796cdf8d4421fb7f9d0f614743bdc: TxnStatusTablet state changed. Reason: New leader 18fb7927ff43460790088c7b6465d2fb. Latest consensus state: current_term: 1 leader_uuid: "18fb7927ff43460790088c7b6465d2fb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1796cdf8d4421fb7f9d0f614743bdc" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 39087 } } peers { permanent_uuid: "18fb7927ff43460790088c7b6465d2fb" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 43831 } } peers { permanent_uuid: "b40e1c9e9a4640108769f458b51d8690" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 45353 } } }
I20260809 16:20:27.679200 25210 tablet_replica.cc:442] T 375a3271b4e2497995b936b9417f0d7a P 4f1796cdf8d4421fb7f9d0f614743bdc: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 16:20:27.679761 25214 tablet_replica.cc:439] T 375a3271b4e2497995b936b9417f0d7a P 18fb7927ff43460790088c7b6465d2fb: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "18fb7927ff43460790088c7b6465d2fb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1796cdf8d4421fb7f9d0f614743bdc" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 39087 } } peers { permanent_uuid: "18fb7927ff43460790088c7b6465d2fb" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 43831 } } peers { permanent_uuid: "b40e1c9e9a4640108769f458b51d8690" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 45353 } } }
I20260809 16:20:27.679855 25214 tablet_replica.cc:442] T 375a3271b4e2497995b936b9417f0d7a P 18fb7927ff43460790088c7b6465d2fb: This TxnStatusTablet replica's current role is: LEADER
I20260809 16:20:27.679922 25238 mvcc.cc:205] Tried to move back new op lower bound from 7316653783758651392 to 7316653783718207488. Current Snapshot: MvccSnapshot[applied={T|T < 7316653783758651392}]
I20260809 16:20:27.680015 25214 tablet_replica.cc:439] T 375a3271b4e2497995b936b9417f0d7a P 18fb7927ff43460790088c7b6465d2fb: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "18fb7927ff43460790088c7b6465d2fb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1796cdf8d4421fb7f9d0f614743bdc" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 39087 } } peers { permanent_uuid: "18fb7927ff43460790088c7b6465d2fb" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 43831 } } peers { permanent_uuid: "b40e1c9e9a4640108769f458b51d8690" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 45353 } } }
I20260809 16:20:27.680095 25214 tablet_replica.cc:442] T 375a3271b4e2497995b936b9417f0d7a P 18fb7927ff43460790088c7b6465d2fb: This TxnStatusTablet replica's current role is: LEADER
I20260809 16:20:27.680397 25213 tablet_replica.cc:439] T 375a3271b4e2497995b936b9417f0d7a P b40e1c9e9a4640108769f458b51d8690: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "18fb7927ff43460790088c7b6465d2fb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1796cdf8d4421fb7f9d0f614743bdc" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 39087 } } peers { permanent_uuid: "18fb7927ff43460790088c7b6465d2fb" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 43831 } } peers { permanent_uuid: "b40e1c9e9a4640108769f458b51d8690" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 45353 } } }
I20260809 16:20:27.680493 25213 tablet_replica.cc:442] T 375a3271b4e2497995b936b9417f0d7a P b40e1c9e9a4640108769f458b51d8690: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 16:20:27.680505 25210 tablet_replica.cc:439] T 375a3271b4e2497995b936b9417f0d7a P 4f1796cdf8d4421fb7f9d0f614743bdc: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "18fb7927ff43460790088c7b6465d2fb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1796cdf8d4421fb7f9d0f614743bdc" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 39087 } } peers { permanent_uuid: "18fb7927ff43460790088c7b6465d2fb" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 43831 } } peers { permanent_uuid: "b40e1c9e9a4640108769f458b51d8690" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 45353 } } }
I20260809 16:20:27.680593 25210 tablet_replica.cc:442] T 375a3271b4e2497995b936b9417f0d7a P 4f1796cdf8d4421fb7f9d0f614743bdc: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 16:20:27.680974 25239 mvcc.cc:205] Tried to move back new op lower bound from 7316653783758651392 to 7316653783718207488. Current Snapshot: MvccSnapshot[applied={T|T < 7316653783758651392}]
I20260809 16:20:27.681355 25244 mvcc.cc:205] Tried to move back new op lower bound from 7316653783758651392 to 7316653783718207488. Current Snapshot: MvccSnapshot[applied={T|T < 7316653783758651392}]
I20260809 16:20:27.685986 25202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:27.683262 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39252 (local address 127.21.51.130:43831)
0809 16:20:27.683305 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0809 16:20:27.683307 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:20:27.683316 (+ 9us) server_negotiation.cc:407] Connection header received
0809 16:20:27.683340 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:20:27.683341 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:20:27.683352 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:20:27.683421 (+ 69us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 16:20:27.683553 (+ 132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:27.684373 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:27.685191 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:27.685540 (+ 349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:27.685621 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:27.685749 (+ 128us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 16:20:27.685787 (+ 38us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 16:20:27.685844 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0809 16:20:27.685888 (+ 44us) server_negotiation.cc:299] Negotiation successful
0809 16:20:27.685911 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260809 16:20:27.691409 25202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:27.688513 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.51.130:60635 (local address 127.21.51.130:43831)
0809 16:20:27.688568 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0809 16:20:27.688570 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:20:27.688751 (+ 181us) server_negotiation.cc:407] Connection header received
0809 16:20:27.688820 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:20:27.688820 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:20:27.688829 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:20:27.688872 (+ 43us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 16:20:27.689019 (+ 147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:27.689437 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:27.690684 (+ 1247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:27.690854 (+ 170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:27.690897 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:27.691140 (+ 243us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 16:20:27.691171 (+ 31us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 16:20:27.691227 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0809 16:20:27.691321 (+ 94us) server_negotiation.cc:299] Negotiation successful
0809 16:20:27.691337 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":33}
I20260809 16:20:27.691530 25257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:27.688424 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.130:43831 (local address 127.21.51.130:60635)
0809 16:20:27.688654 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0809 16:20:27.688665 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 16:20:27.688756 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:20:27.688876 (+ 120us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:20:27.688877 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:20:27.688879 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 16:20:27.688950 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:20:27.688951 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:20:27.689517 (+ 566us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:20:27.689518 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:20:27.690592 (+ 1074us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:20:27.690595 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:20:27.690924 (+ 329us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:20:27.690925 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:20:27.690998 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:27.691013 (+ 15us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 16:20:27.691258 (+ 245us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 16:20:27.691260 (+ 2us) client_negotiation.cc:769] Sending connection context
0809 16:20:27.691316 (+ 56us) client_negotiation.cc:240] Negotiation successful
0809 16:20:27.691362 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":181,"thread_start_us":103,"threads_started":1}
I20260809 16:20:27.694336 25091 raft_consensus.cc:1261] T 999a50f911744834bd484eb981ce1348 P b40e1c9e9a4640108769f458b51d8690 [term 1 FOLLOWER]: Refusing update from remote peer 18fb7927ff43460790088c7b6465d2fb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 16:20:27.694451 24647 raft_consensus.cc:1261] T 999a50f911744834bd484eb981ce1348 P 4f1796cdf8d4421fb7f9d0f614743bdc [term 1 FOLLOWER]: Refusing update from remote peer 18fb7927ff43460790088c7b6465d2fb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 16:20:27.694522 25214 consensus_queue.cc:1035] T 999a50f911744834bd484eb981ce1348 P 18fb7927ff43460790088c7b6465d2fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "b40e1c9e9a4640108769f458b51d8690" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 45353 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 16:20:27.694756 25214 consensus_queue.cc:1035] T 999a50f911744834bd484eb981ce1348 P 18fb7927ff43460790088c7b6465d2fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1796cdf8d4421fb7f9d0f614743bdc" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 39087 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 16:20:27.707360 24411 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60116:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 16:20:27.707476 24411 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60116:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 16:20:27.708233 24411 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4d1cbb0203184af5a564b1a0deb4dd3f: Sending DeleteTablet for 3 replicas of tablet 999a50f911744834bd484eb981ce1348
I20260809 16:20:27.708514 25060 tablet_service.cc:1514] Processing DeleteTablet for tablet 999a50f911744834bd484eb981ce1348 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:20:27 UTC) from {username='kudu', principal='kudu/127.21.51.190@KRBTEST.COM'} at 127.0.0.1:35402
I20260809 16:20:27.708566 24613 tablet_service.cc:1514] Processing DeleteTablet for tablet 999a50f911744834bd484eb981ce1348 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:20:27 UTC) from {username='kudu', principal='kudu/127.21.51.190@KRBTEST.COM'} at 127.0.0.1:44898
I20260809 16:20:27.708762 24835 tablet_service.cc:1514] Processing DeleteTablet for tablet 999a50f911744834bd484eb981ce1348 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:20:27 UTC) from {username='kudu', principal='kudu/127.21.51.190@KRBTEST.COM'} at 127.0.0.1:39228
I20260809 16:20:27.708830 25268 tablet_replica.cc:330] T 999a50f911744834bd484eb981ce1348 P 4f1796cdf8d4421fb7f9d0f614743bdc: stopping tablet replica
I20260809 16:20:27.708940 25268 raft_consensus.cc:2229] T 999a50f911744834bd484eb981ce1348 P 4f1796cdf8d4421fb7f9d0f614743bdc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 16:20:27.709013 25268 raft_consensus.cc:2258] T 999a50f911744834bd484eb981ce1348 P 4f1796cdf8d4421fb7f9d0f614743bdc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 16:20:27.709095 25269 tablet_replica.cc:330] T 999a50f911744834bd484eb981ce1348 P 18fb7927ff43460790088c7b6465d2fb: stopping tablet replica
I20260809 16:20:27.709326 25266 tablet_replica.cc:330] T 999a50f911744834bd484eb981ce1348 P b40e1c9e9a4640108769f458b51d8690: stopping tablet replica
I20260809 16:20:27.709280 24411 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60116:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 16:20:27.709421 24411 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 16:20:27.709436 25266 raft_consensus.cc:2229] T 999a50f911744834bd484eb981ce1348 P b40e1c9e9a4640108769f458b51d8690 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 16:20:27.709492 25266 raft_consensus.cc:2258] T 999a50f911744834bd484eb981ce1348 P b40e1c9e9a4640108769f458b51d8690 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 16:20:27.709532 25268 ts_tablet_manager.cc:1905] T 999a50f911744834bd484eb981ce1348 P 4f1796cdf8d4421fb7f9d0f614743bdc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 16:20:27.710354 25269 raft_consensus.cc:2229] T 999a50f911744834bd484eb981ce1348 P 18fb7927ff43460790088c7b6465d2fb [term 1 LEADER]: Raft consensus shutting down.
I20260809 16:20:27.710521 25269 raft_consensus.cc:2258] T 999a50f911744834bd484eb981ce1348 P 18fb7927ff43460790088c7b6465d2fb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 16:20:27.711016 25269 ts_tablet_manager.cc:1905] T 999a50f911744834bd484eb981ce1348 P 18fb7927ff43460790088c7b6465d2fb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 16:20:27.711621 24613 tablet_service.cc:1467] Processing CreateTablet for tablet bfe580e0f63a487fad7f2005e1baee49 (DEFAULT_TABLE table=test-table [id=6593bcfb33d544149144a6139d337f89]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:20:27.711824 24613 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bfe580e0f63a487fad7f2005e1baee49. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:20:27.711831 25268 ts_tablet_manager.cc:1918] T 999a50f911744834bd484eb981ce1348 P 4f1796cdf8d4421fb7f9d0f614743bdc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 16:20:27.711941 25268 log.cc:1196] T 999a50f911744834bd484eb981ce1348 P 4f1796cdf8d4421fb7f9d0f614743bdc: Deleting WAL directory at /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786292417722232-21710-0/minicluster-data/ts-0/wal/wals/999a50f911744834bd484eb981ce1348
I20260809 16:20:27.712278 25268 ts_tablet_manager.cc:1939] T 999a50f911744834bd484eb981ce1348 P 4f1796cdf8d4421fb7f9d0f614743bdc: Deleting consensus metadata
I20260809 16:20:27.712738 24388 catalog_manager.cc:4841] TS 4f1796cdf8d4421fb7f9d0f614743bdc (127.21.51.129:39087): tablet 999a50f911744834bd484eb981ce1348 (table test-table [id=93275e6ab7d0476bb1768c7e4c62771e]) successfully deleted
I20260809 16:20:27.713178 25060 tablet_service.cc:1467] Processing CreateTablet for tablet bfe580e0f63a487fad7f2005e1baee49 (DEFAULT_TABLE table=test-table [id=6593bcfb33d544149144a6139d337f89]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:20:27.713192 25269 ts_tablet_manager.cc:1918] T 999a50f911744834bd484eb981ce1348 P 18fb7927ff43460790088c7b6465d2fb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 16:20:27.713317 25060 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bfe580e0f63a487fad7f2005e1baee49. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:20:27.713697 25269 log.cc:1196] T 999a50f911744834bd484eb981ce1348 P 18fb7927ff43460790088c7b6465d2fb: Deleting WAL directory at /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786292417722232-21710-0/minicluster-data/ts-1/wal/wals/999a50f911744834bd484eb981ce1348
I20260809 16:20:27.713968 25269 ts_tablet_manager.cc:1939] T 999a50f911744834bd484eb981ce1348 P 18fb7927ff43460790088c7b6465d2fb: Deleting consensus metadata
I20260809 16:20:27.714696 25205 tablet_bootstrap.cc:492] T bfe580e0f63a487fad7f2005e1baee49 P 4f1796cdf8d4421fb7f9d0f614743bdc: Bootstrap starting.
I20260809 16:20:27.714691 25203 tablet_bootstrap.cc:492] T bfe580e0f63a487fad7f2005e1baee49 P b40e1c9e9a4640108769f458b51d8690: Bootstrap starting.
I20260809 16:20:27.715121 25203 tablet_bootstrap.cc:654] T bfe580e0f63a487fad7f2005e1baee49 P b40e1c9e9a4640108769f458b51d8690: Neither blocks nor log segments found. Creating new log.
I20260809 16:20:27.715131 25205 tablet_bootstrap.cc:654] T bfe580e0f63a487fad7f2005e1baee49 P 4f1796cdf8d4421fb7f9d0f614743bdc: Neither blocks nor log segments found. Creating new log.
I20260809 16:20:27.715670 24835 tablet_service.cc:1467] Processing CreateTablet for tablet bfe580e0f63a487fad7f2005e1baee49 (DEFAULT_TABLE table=test-table [id=6593bcfb33d544149144a6139d337f89]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:20:27.715791 24835 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bfe580e0f63a487fad7f2005e1baee49. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:20:27.715827 25203 tablet_bootstrap.cc:492] T bfe580e0f63a487fad7f2005e1baee49 P b40e1c9e9a4640108769f458b51d8690: No bootstrap required, opened a new log
I20260809 16:20:27.715883 25203 ts_tablet_manager.cc:1397] T bfe580e0f63a487fad7f2005e1baee49 P b40e1c9e9a4640108769f458b51d8690: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260809 16:20:27.716027 25203 raft_consensus.cc:348] T bfe580e0f63a487fad7f2005e1baee49 P b40e1c9e9a4640108769f458b51d8690 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1796cdf8d4421fb7f9d0f614743bdc" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 39087 } } peers { permanent_uuid: "18fb7927ff43460790088c7b6465d2fb" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 43831 } } peers { permanent_uuid: "b40e1c9e9a4640108769f458b51d8690" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 45353 } }
I20260809 16:20:27.716107 25203 raft_consensus.cc:374] T bfe580e0f63a487fad7f2005e1baee49 P b40e1c9e9a4640108769f458b51d8690 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:20:27.716136 25203 raft_consensus.cc:729] T bfe580e0f63a487fad7f2005e1baee49 P b40e1c9e9a4640108769f458b51d8690 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b40e1c9e9a4640108769f458b51d8690, State: Initialized, Role: FOLLOWER
I20260809 16:20:27.716177 25203 consensus_queue.cc:260] T bfe580e0f63a487fad7f2005e1baee49 P b40e1c9e9a4640108769f458b51d8690 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1796cdf8d4421fb7f9d0f614743bdc" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 39087 } } peers { permanent_uuid: "18fb7927ff43460790088c7b6465d2fb" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 43831 } } peers { permanent_uuid: "b40e1c9e9a4640108769f458b51d8690" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 45353 } }
I20260809 16:20:27.716269 25203 ts_tablet_manager.cc:1428] T bfe580e0f63a487fad7f2005e1baee49 P b40e1c9e9a4640108769f458b51d8690: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260809 16:20:27.716945 24389 catalog_manager.cc:4841] TS 18fb7927ff43460790088c7b6465d2fb (127.21.51.130:43831): tablet 999a50f911744834bd484eb981ce1348 (table test-table [id=93275e6ab7d0476bb1768c7e4c62771e]) successfully deleted
I20260809 16:20:27.717082 25206 tablet_bootstrap.cc:492] T bfe580e0f63a487fad7f2005e1baee49 P 18fb7927ff43460790088c7b6465d2fb: Bootstrap starting.
I20260809 16:20:27.717299 25205 tablet_bootstrap.cc:492] T bfe580e0f63a487fad7f2005e1baee49 P 4f1796cdf8d4421fb7f9d0f614743bdc: No bootstrap required, opened a new log
I20260809 16:20:27.717365 25205 ts_tablet_manager.cc:1397] T bfe580e0f63a487fad7f2005e1baee49 P 4f1796cdf8d4421fb7f9d0f614743bdc: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260809 16:20:27.717530 25206 tablet_bootstrap.cc:654] T bfe580e0f63a487fad7f2005e1baee49 P 18fb7927ff43460790088c7b6465d2fb: Neither blocks nor log segments found. Creating new log.
I20260809 16:20:27.717516 25205 raft_consensus.cc:348] T bfe580e0f63a487fad7f2005e1baee49 P 4f1796cdf8d4421fb7f9d0f614743bdc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1796cdf8d4421fb7f9d0f614743bdc" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 39087 } } peers { permanent_uuid: "18fb7927ff43460790088c7b6465d2fb" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 43831 } } peers { permanent_uuid: "b40e1c9e9a4640108769f458b51d8690" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 45353 } }
I20260809 16:20:27.717592 25205 raft_consensus.cc:374] T bfe580e0f63a487fad7f2005e1baee49 P 4f1796cdf8d4421fb7f9d0f614743bdc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:20:27.717622 25205 raft_consensus.cc:729] T bfe580e0f63a487fad7f2005e1baee49 P 4f1796cdf8d4421fb7f9d0f614743bdc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f1796cdf8d4421fb7f9d0f614743bdc, State: Initialized, Role: FOLLOWER
I20260809 16:20:27.717671 25205 consensus_queue.cc:260] T bfe580e0f63a487fad7f2005e1baee49 P 4f1796cdf8d4421fb7f9d0f614743bdc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1796cdf8d4421fb7f9d0f614743bdc" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 39087 } } peers { permanent_uuid: "18fb7927ff43460790088c7b6465d2fb" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 43831 } } peers { permanent_uuid: "b40e1c9e9a4640108769f458b51d8690" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 45353 } }
I20260809 16:20:27.717768 25205 ts_tablet_manager.cc:1428] T bfe580e0f63a487fad7f2005e1baee49 P 4f1796cdf8d4421fb7f9d0f614743bdc: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260809 16:20:27.717938 25266 ts_tablet_manager.cc:1905] T 999a50f911744834bd484eb981ce1348 P b40e1c9e9a4640108769f458b51d8690: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 16:20:27.719151 25206 tablet_bootstrap.cc:492] T bfe580e0f63a487fad7f2005e1baee49 P 18fb7927ff43460790088c7b6465d2fb: No bootstrap required, opened a new log
I20260809 16:20:27.719201 25206 ts_tablet_manager.cc:1397] T bfe580e0f63a487fad7f2005e1baee49 P 18fb7927ff43460790088c7b6465d2fb: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 16:20:27.719403 25206 raft_consensus.cc:348] T bfe580e0f63a487fad7f2005e1baee49 P 18fb7927ff43460790088c7b6465d2fb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1796cdf8d4421fb7f9d0f614743bdc" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 39087 } } peers { permanent_uuid: "18fb7927ff43460790088c7b6465d2fb" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 43831 } } peers { permanent_uuid: "b40e1c9e9a4640108769f458b51d8690" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 45353 } }
I20260809 16:20:27.719472 25206 raft_consensus.cc:374] T bfe580e0f63a487fad7f2005e1baee49 P 18fb7927ff43460790088c7b6465d2fb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:20:27.719468 25266 ts_tablet_manager.cc:1918] T 999a50f911744834bd484eb981ce1348 P b40e1c9e9a4640108769f458b51d8690: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 16:20:27.719494 25206 raft_consensus.cc:729] T bfe580e0f63a487fad7f2005e1baee49 P 18fb7927ff43460790088c7b6465d2fb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 18fb7927ff43460790088c7b6465d2fb, State: Initialized, Role: FOLLOWER
I20260809 16:20:27.719530 25266 log.cc:1196] T 999a50f911744834bd484eb981ce1348 P b40e1c9e9a4640108769f458b51d8690: Deleting WAL directory at /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786292417722232-21710-0/minicluster-data/ts-2/wal/wals/999a50f911744834bd484eb981ce1348
I20260809 16:20:27.719542 25206 consensus_queue.cc:260] T bfe580e0f63a487fad7f2005e1baee49 P 18fb7927ff43460790088c7b6465d2fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1796cdf8d4421fb7f9d0f614743bdc" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 39087 } } peers { permanent_uuid: "18fb7927ff43460790088c7b6465d2fb" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 43831 } } peers { permanent_uuid: "b40e1c9e9a4640108769f458b51d8690" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 45353 } }
I20260809 16:20:27.719662 25206 ts_tablet_manager.cc:1428] T bfe580e0f63a487fad7f2005e1baee49 P 18fb7927ff43460790088c7b6465d2fb: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 16:20:27.719820 25266 ts_tablet_manager.cc:1939] T 999a50f911744834bd484eb981ce1348 P b40e1c9e9a4640108769f458b51d8690: Deleting consensus metadata
I20260809 16:20:27.720250 24389 catalog_manager.cc:4841] TS b40e1c9e9a4640108769f458b51d8690 (127.21.51.131:45353): tablet 999a50f911744834bd484eb981ce1348 (table test-table [id=93275e6ab7d0476bb1768c7e4c62771e]) successfully deleted
I20260809 16:20:27.962155 25213 raft_consensus.cc:482] T bfe580e0f63a487fad7f2005e1baee49 P b40e1c9e9a4640108769f458b51d8690 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 16:20:27.962281 25213 raft_consensus.cc:504] T bfe580e0f63a487fad7f2005e1baee49 P b40e1c9e9a4640108769f458b51d8690 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1796cdf8d4421fb7f9d0f614743bdc" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 39087 } } peers { permanent_uuid: "18fb7927ff43460790088c7b6465d2fb" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 43831 } } peers { permanent_uuid: "b40e1c9e9a4640108769f458b51d8690" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 45353 } }
I20260809 16:20:27.962647 25213 leader_election.cc:288] T bfe580e0f63a487fad7f2005e1baee49 P b40e1c9e9a4640108769f458b51d8690 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4f1796cdf8d4421fb7f9d0f614743bdc (127.21.51.129:39087), 18fb7927ff43460790088c7b6465d2fb (127.21.51.130:43831)
I20260809 16:20:27.966167 25278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:27.962851 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.129:39087 (local address 127.21.51.131:40317)
0809 16:20:27.963016 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0809 16:20:27.963029 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 16:20:27.963124 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:20:27.963292 (+ 168us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:20:27.963293 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:20:27.963296 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:20:27.963392 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:20:27.963394 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:20:27.964213 (+ 819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:20:27.964214 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:20:27.965286 (+ 1072us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:20:27.965289 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:20:27.965906 (+ 617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:20:27.965907 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:20:27.965980 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:27.965986 (+ 6us) client_negotiation.cc:769] Sending connection context
0809 16:20:27.966041 (+ 55us) client_negotiation.cc:240] Negotiation successful
0809 16:20:27.966090 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":70,"threads_started":1}
I20260809 16:20:27.966197 25200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:27.962940 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.51.131:40317 (local address 127.21.51.129:39087)
0809 16:20:27.962993 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0809 16:20:27.962996 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 16:20:27.963106 (+ 110us) server_negotiation.cc:407] Connection header received
0809 16:20:27.963203 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:20:27.963204 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:20:27.963214 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:20:27.963278 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:20:27.963500 (+ 222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:27.964089 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:27.965371 (+ 1282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:27.965825 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:27.965881 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:27.965889 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0809 16:20:27.966065 (+ 176us) server_negotiation.cc:299] Negotiation successful
0809 16:20:27.966113 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260809 16:20:27.966372 25202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:27.962993 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.51.131:58743 (local address 127.21.51.130:43831)
0809 16:20:27.963051 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0809 16:20:27.963054 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 16:20:27.963207 (+ 153us) server_negotiation.cc:407] Connection header received
0809 16:20:27.963346 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:20:27.963347 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:20:27.963360 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:20:27.963444 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:20:27.963622 (+ 178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:27.964298 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:27.965547 (+ 1249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:27.965948 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:27.966029 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:27.966042 (+ 13us) server_negotiation.cc:1035] Waiting for connection context
0809 16:20:27.966233 (+ 191us) server_negotiation.cc:299] Negotiation successful
0809 16:20:27.966278 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260809 16:20:27.966357 25279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:27.962896 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.130:43831 (local address 127.21.51.131:58743)
0809 16:20:27.963101 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0809 16:20:27.963114 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 16:20:27.963203 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:20:27.963457 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:20:27.963458 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:20:27.963461 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:20:27.963532 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:20:27.963534 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:20:27.964422 (+ 888us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:20:27.964423 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:20:27.965459 (+ 1036us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:20:27.965462 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:20:27.966073 (+ 611us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:20:27.966074 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:20:27.966146 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:27.966150 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 16:20:27.966225 (+ 75us) client_negotiation.cc:240] Negotiation successful
0809 16:20:27.966271 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":95,"threads_started":1}
I20260809 16:20:27.966856 24878 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bfe580e0f63a487fad7f2005e1baee49" candidate_uuid: "b40e1c9e9a4640108769f458b51d8690" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "18fb7927ff43460790088c7b6465d2fb" is_pre_election: true
I20260809 16:20:27.966997 24878 raft_consensus.cc:2454] T bfe580e0f63a487fad7f2005e1baee49 P 18fb7927ff43460790088c7b6465d2fb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b40e1c9e9a4640108769f458b51d8690 in term 0.
I20260809 16:20:27.967140 24647 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bfe580e0f63a487fad7f2005e1baee49" candidate_uuid: "b40e1c9e9a4640108769f458b51d8690" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f1796cdf8d4421fb7f9d0f614743bdc" is_pre_election: true
I20260809 16:20:27.967197 24987 leader_election.cc:302] T bfe580e0f63a487fad7f2005e1baee49 P b40e1c9e9a4640108769f458b51d8690 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 18fb7927ff43460790088c7b6465d2fb, b40e1c9e9a4640108769f458b51d8690; no voters:
I20260809 16:20:27.967244 24647 raft_consensus.cc:2454] T bfe580e0f63a487fad7f2005e1baee49 P 4f1796cdf8d4421fb7f9d0f614743bdc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b40e1c9e9a4640108769f458b51d8690 in term 0.
I20260809 16:20:27.967396 25213 raft_consensus.cc:2781] T bfe580e0f63a487fad7f2005e1baee49 P b40e1c9e9a4640108769f458b51d8690 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 16:20:27.967447 25213 raft_consensus.cc:482] T bfe580e0f63a487fad7f2005e1baee49 P b40e1c9e9a4640108769f458b51d8690 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 16:20:27.967463 25213 raft_consensus.cc:3037] T bfe580e0f63a487fad7f2005e1baee49 P b40e1c9e9a4640108769f458b51d8690 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:20:27.968266 25213 raft_consensus.cc:504] T bfe580e0f63a487fad7f2005e1baee49 P b40e1c9e9a4640108769f458b51d8690 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1796cdf8d4421fb7f9d0f614743bdc" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 39087 } } peers { permanent_uuid: "18fb7927ff43460790088c7b6465d2fb" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 43831 } } peers { permanent_uuid: "b40e1c9e9a4640108769f458b51d8690" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 45353 } }
I20260809 16:20:27.968417 25213 leader_election.cc:288] T bfe580e0f63a487fad7f2005e1baee49 P b40e1c9e9a4640108769f458b51d8690 [CANDIDATE]: Term 1 election: Requested vote from peers 4f1796cdf8d4421fb7f9d0f614743bdc (127.21.51.129:39087), 18fb7927ff43460790088c7b6465d2fb (127.21.51.130:43831)
I20260809 16:20:27.968638 24647 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bfe580e0f63a487fad7f2005e1baee49" candidate_uuid: "b40e1c9e9a4640108769f458b51d8690" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f1796cdf8d4421fb7f9d0f614743bdc"
I20260809 16:20:27.968626 24878 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bfe580e0f63a487fad7f2005e1baee49" candidate_uuid: "b40e1c9e9a4640108769f458b51d8690" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "18fb7927ff43460790088c7b6465d2fb"
I20260809 16:20:27.968742 24647 raft_consensus.cc:3037] T bfe580e0f63a487fad7f2005e1baee49 P 4f1796cdf8d4421fb7f9d0f614743bdc [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:20:27.968750 24878 raft_consensus.cc:3037] T bfe580e0f63a487fad7f2005e1baee49 P 18fb7927ff43460790088c7b6465d2fb [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:20:27.969421 24878 raft_consensus.cc:2454] T bfe580e0f63a487fad7f2005e1baee49 P 18fb7927ff43460790088c7b6465d2fb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b40e1c9e9a4640108769f458b51d8690 in term 1.
I20260809 16:20:27.969565 24987 leader_election.cc:302] T bfe580e0f63a487fad7f2005e1baee49 P b40e1c9e9a4640108769f458b51d8690 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 18fb7927ff43460790088c7b6465d2fb, b40e1c9e9a4640108769f458b51d8690; no voters:
I20260809 16:20:27.969568 24647 raft_consensus.cc:2454] T bfe580e0f63a487fad7f2005e1baee49 P 4f1796cdf8d4421fb7f9d0f614743bdc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b40e1c9e9a4640108769f458b51d8690 in term 1.
I20260809 16:20:27.969659 25213 raft_consensus.cc:2781] T bfe580e0f63a487fad7f2005e1baee49 P b40e1c9e9a4640108769f458b51d8690 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:20:27.969771 25213 raft_consensus.cc:686] T bfe580e0f63a487fad7f2005e1baee49 P b40e1c9e9a4640108769f458b51d8690 [term 1 LEADER]: Becoming Leader. State: Replica: b40e1c9e9a4640108769f458b51d8690, State: Running, Role: LEADER
I20260809 16:20:27.969851 25213 consensus_queue.cc:237] T bfe580e0f63a487fad7f2005e1baee49 P b40e1c9e9a4640108769f458b51d8690 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1796cdf8d4421fb7f9d0f614743bdc" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 39087 } } peers { permanent_uuid: "18fb7927ff43460790088c7b6465d2fb" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 43831 } } peers { permanent_uuid: "b40e1c9e9a4640108769f458b51d8690" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 45353 } }
I20260809 16:20:27.970580 24411 catalog_manager.cc:5495] T bfe580e0f63a487fad7f2005e1baee49 P b40e1c9e9a4640108769f458b51d8690 reported cstate change: term changed from 0 to 1, leader changed from <none> to b40e1c9e9a4640108769f458b51d8690 (127.21.51.131). New cstate: current_term: 1 leader_uuid: "b40e1c9e9a4640108769f458b51d8690" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1796cdf8d4421fb7f9d0f614743bdc" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 39087 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "18fb7927ff43460790088c7b6465d2fb" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 43831 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b40e1c9e9a4640108769f458b51d8690" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 45353 } health_report { overall_health: HEALTHY } } }
I20260809 16:20:28.008769 25196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:28.006320 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35410 (local address 127.21.51.131:45353)
0809 16:20:28.006362 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0809 16:20:28.006364 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:20:28.006375 (+ 11us) server_negotiation.cc:407] Connection header received
0809 16:20:28.006417 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:20:28.006418 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:20:28.006439 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:20:28.006534 (+ 95us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 16:20:28.006652 (+ 118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:28.007146 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:28.007891 (+ 745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:28.008110 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:28.008185 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:28.008351 (+ 166us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 16:20:28.008471 (+ 120us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 16:20:28.008555 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0809 16:20:28.008599 (+ 44us) server_negotiation.cc:299] Negotiation successful
0809 16:20:28.008623 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260809 16:20:28.013958 25202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:28.011405 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.51.131:33941 (local address 127.21.51.130:43831)
0809 16:20:28.011484 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0809 16:20:28.011486 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:20:28.011593 (+ 107us) server_negotiation.cc:407] Connection header received
0809 16:20:28.011656 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:20:28.011657 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:20:28.011668 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:20:28.011771 (+ 103us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 16:20:28.011920 (+ 149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:28.012387 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:28.013214 (+ 827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:28.013416 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:28.013491 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:28.013667 (+ 176us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 16:20:28.013710 (+ 43us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 16:20:28.013770 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0809 16:20:28.013827 (+ 57us) server_negotiation.cc:299] Negotiation successful
0809 16:20:28.013846 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260809 16:20:28.013986 25288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:28.011301 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.130:43831 (local address 127.21.51.131:33941)
0809 16:20:28.011510 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0809 16:20:28.011522 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 16:20:28.011594 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:20:28.011780 (+ 186us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:20:28.011781 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:20:28.011782 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 16:20:28.011842 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:20:28.011843 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:20:28.012473 (+ 630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:20:28.012474 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:20:28.013109 (+ 635us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:20:28.013111 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:20:28.013492 (+ 381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:20:28.013493 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:20:28.013560 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:28.013577 (+ 17us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 16:20:28.013773 (+ 196us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 16:20:28.013776 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 16:20:28.013824 (+ 48us) client_negotiation.cc:240] Negotiation successful
0809 16:20:28.013891 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":92,"threads_started":1}
I20260809 16:20:28.016136 24647 raft_consensus.cc:1261] T bfe580e0f63a487fad7f2005e1baee49 P 4f1796cdf8d4421fb7f9d0f614743bdc [term 1 FOLLOWER]: Refusing update from remote peer b40e1c9e9a4640108769f458b51d8690: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 16:20:28.016260 24878 raft_consensus.cc:1261] T bfe580e0f63a487fad7f2005e1baee49 P 18fb7927ff43460790088c7b6465d2fb [term 1 FOLLOWER]: Refusing update from remote peer b40e1c9e9a4640108769f458b51d8690: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 16:20:28.016399 25213 consensus_queue.cc:1035] T bfe580e0f63a487fad7f2005e1baee49 P b40e1c9e9a4640108769f458b51d8690 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1796cdf8d4421fb7f9d0f614743bdc" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 39087 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 16:20:28.016764 25213 consensus_queue.cc:1035] T bfe580e0f63a487fad7f2005e1baee49 P b40e1c9e9a4640108769f458b51d8690 [LEADER]: Connected to new peer: Peer: permanent_uuid: "18fb7927ff43460790088c7b6465d2fb" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 43831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 16:20:28.023597 25196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:28.021338 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.51.130:54693 (local address 127.21.51.131:45353)
0809 16:20:28.021380 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0809 16:20:28.021383 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 16:20:28.021392 (+ 9us) server_negotiation.cc:407] Connection header received
0809 16:20:28.021438 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:20:28.021439 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:20:28.021452 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:20:28.021505 (+ 53us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 16:20:28.021602 (+ 97us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:28.022062 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:28.022880 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:28.023159 (+ 279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:28.023226 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:28.023387 (+ 161us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 16:20:28.023418 (+ 31us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 16:20:28.023469 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0809 16:20:28.023524 (+ 55us) server_negotiation.cc:299] Negotiation successful
0809 16:20:28.023551 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260809 16:20:28.023629 25257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:28.021283 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.131:45353 (local address 127.21.51.130:54693)
0809 16:20:28.021333 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0809 16:20:28.021344 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 16:20:28.021383 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:20:28.021509 (+ 126us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:20:28.021510 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:20:28.021511 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 16:20:28.021552 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:20:28.021553 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:20:28.022129 (+ 576us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:20:28.022129 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:20:28.022766 (+ 637us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:20:28.022769 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:20:28.023237 (+ 468us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:20:28.023237 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:20:28.023317 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:28.023325 (+ 8us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 16:20:28.023469 (+ 144us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 16:20:28.023471 (+ 2us) client_negotiation.cc:769] Sending connection context
0809 16:20:28.023511 (+ 40us) client_negotiation.cc:240] Negotiation successful
0809 16:20:28.023562 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31}
W20260809 16:20:28.023943 25239 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260809 16:20:29.543926 24411 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60116:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 16:20:29.544026 24411 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60116:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 16:20:29.544998 24411 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4d1cbb0203184af5a564b1a0deb4dd3f: Sending DeleteTablet for 3 replicas of tablet bfe580e0f63a487fad7f2005e1baee49
I20260809 16:20:29.545279 24613 tablet_service.cc:1514] Processing DeleteTablet for tablet bfe580e0f63a487fad7f2005e1baee49 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:20:29 UTC) from {username='kudu', principal='kudu/127.21.51.190@KRBTEST.COM'} at 127.0.0.1:44898
I20260809 16:20:29.545315 25060 tablet_service.cc:1514] Processing DeleteTablet for tablet bfe580e0f63a487fad7f2005e1baee49 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:20:29 UTC) from {username='kudu', principal='kudu/127.21.51.190@KRBTEST.COM'} at 127.0.0.1:35402
I20260809 16:20:29.545470 24835 tablet_service.cc:1514] Processing DeleteTablet for tablet bfe580e0f63a487fad7f2005e1baee49 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:20:29 UTC) from {username='kudu', principal='kudu/127.21.51.190@KRBTEST.COM'} at 127.0.0.1:39228
I20260809 16:20:29.545680 25328 tablet_replica.cc:330] T bfe580e0f63a487fad7f2005e1baee49 P 18fb7927ff43460790088c7b6465d2fb: stopping tablet replica
I20260809 16:20:29.545763 25328 raft_consensus.cc:2229] T bfe580e0f63a487fad7f2005e1baee49 P 18fb7927ff43460790088c7b6465d2fb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 16:20:29.545847 25328 raft_consensus.cc:2258] T bfe580e0f63a487fad7f2005e1baee49 P 18fb7927ff43460790088c7b6465d2fb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 16:20:29.546172 25328 ts_tablet_manager.cc:1905] T bfe580e0f63a487fad7f2005e1baee49 P 18fb7927ff43460790088c7b6465d2fb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 16:20:29.548336 25328 ts_tablet_manager.cc:1918] T bfe580e0f63a487fad7f2005e1baee49 P 18fb7927ff43460790088c7b6465d2fb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260809 16:20:29.548487 25328 log.cc:1196] T bfe580e0f63a487fad7f2005e1baee49 P 18fb7927ff43460790088c7b6465d2fb: Deleting WAL directory at /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786292417722232-21710-0/minicluster-data/ts-1/wal/wals/bfe580e0f63a487fad7f2005e1baee49
I20260809 16:20:29.548875 25328 ts_tablet_manager.cc:1939] T bfe580e0f63a487fad7f2005e1baee49 P 18fb7927ff43460790088c7b6465d2fb: Deleting consensus metadata
I20260809 16:20:29.548964 25326 tablet_replica.cc:330] T bfe580e0f63a487fad7f2005e1baee49 P 4f1796cdf8d4421fb7f9d0f614743bdc: stopping tablet replica
I20260809 16:20:29.549054 25326 raft_consensus.cc:2229] T bfe580e0f63a487fad7f2005e1baee49 P 4f1796cdf8d4421fb7f9d0f614743bdc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 16:20:29.549101 25326 raft_consensus.cc:2258] T bfe580e0f63a487fad7f2005e1baee49 P 4f1796cdf8d4421fb7f9d0f614743bdc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 16:20:29.549234 25325 tablet_replica.cc:330] T bfe580e0f63a487fad7f2005e1baee49 P b40e1c9e9a4640108769f458b51d8690: stopping tablet replica
I20260809 16:20:29.549232 24389 catalog_manager.cc:4841] TS 18fb7927ff43460790088c7b6465d2fb (127.21.51.130:43831): tablet bfe580e0f63a487fad7f2005e1baee49 (table test-table [id=6593bcfb33d544149144a6139d337f89]) successfully deleted
I20260809 16:20:29.549299 25325 raft_consensus.cc:2229] T bfe580e0f63a487fad7f2005e1baee49 P b40e1c9e9a4640108769f458b51d8690 [term 1 LEADER]: Raft consensus shutting down.
I20260809 16:20:29.549489 25326 ts_tablet_manager.cc:1905] T bfe580e0f63a487fad7f2005e1baee49 P 4f1796cdf8d4421fb7f9d0f614743bdc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 16:20:29.550678 25326 ts_tablet_manager.cc:1918] T bfe580e0f63a487fad7f2005e1baee49 P 4f1796cdf8d4421fb7f9d0f614743bdc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260809 16:20:29.550750 25326 log.cc:1196] T bfe580e0f63a487fad7f2005e1baee49 P 4f1796cdf8d4421fb7f9d0f614743bdc: Deleting WAL directory at /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786292417722232-21710-0/minicluster-data/ts-0/wal/wals/bfe580e0f63a487fad7f2005e1baee49
I20260809 16:20:29.550974 25326 ts_tablet_manager.cc:1939] T bfe580e0f63a487fad7f2005e1baee49 P 4f1796cdf8d4421fb7f9d0f614743bdc: Deleting consensus metadata
I20260809 16:20:29.551260 25325 raft_consensus.cc:2258] T bfe580e0f63a487fad7f2005e1baee49 P b40e1c9e9a4640108769f458b51d8690 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 16:20:29.551324 24388 catalog_manager.cc:4841] TS 4f1796cdf8d4421fb7f9d0f614743bdc (127.21.51.129:39087): tablet bfe580e0f63a487fad7f2005e1baee49 (table test-table [id=6593bcfb33d544149144a6139d337f89]) successfully deleted
I20260809 16:20:29.551545 25325 ts_tablet_manager.cc:1905] T bfe580e0f63a487fad7f2005e1baee49 P b40e1c9e9a4640108769f458b51d8690: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 16:20:29.553086 25325 ts_tablet_manager.cc:1918] T bfe580e0f63a487fad7f2005e1baee49 P b40e1c9e9a4640108769f458b51d8690: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260809 16:20:29.553144 25325 log.cc:1196] T bfe580e0f63a487fad7f2005e1baee49 P b40e1c9e9a4640108769f458b51d8690: Deleting WAL directory at /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786292417722232-21710-0/minicluster-data/ts-2/wal/wals/bfe580e0f63a487fad7f2005e1baee49
I20260809 16:20:29.553403 25325 ts_tablet_manager.cc:1939] T bfe580e0f63a487fad7f2005e1baee49 P b40e1c9e9a4640108769f458b51d8690: Deleting consensus metadata
I20260809 16:20:29.553730 24389 catalog_manager.cc:4841] TS b40e1c9e9a4640108769f458b51d8690 (127.21.51.131:45353): tablet bfe580e0f63a487fad7f2005e1baee49 (table test-table [id=6593bcfb33d544149144a6139d337f89]) successfully deleted
Aug 09 16:20:29 dist-test-slave-s11t krb5kdc[24339](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292429, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 09 16:20:29 dist-test-slave-s11t krb5kdc[24339](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292429, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.21.51.190@KRBTEST.COM
I20260809 16:20:29.579224 25339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:29.565329 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60128 (local address 127.21.51.190:38211)
0809 16:20:29.565481 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0809 16:20:29.565484 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 16:20:29.565563 (+ 79us) server_negotiation.cc:407] Connection header received
0809 16:20:29.565687 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:20:29.565688 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:20:29.565701 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:20:29.565793 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:20:29.566444 (+ 651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:29.567012 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:29.567881 (+ 869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:29.568110 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:29.568193 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:29.576455 (+ 8262us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:20:29.576457 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:20:29.576457 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:20:29.576465 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:20:29.578094 (+ 1629us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:20:29.578440 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:20:29.578441 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:20:29.578441 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:20:29.578515 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:20:29.578755 (+ 240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:20:29.578756 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:20:29.578756 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:20:29.578874 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:20:29.578953 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0809 16:20:29.579058 (+ 105us) server_negotiation.cc:299] Negotiation successful
0809 16:20:29.579124 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":121,"thread_start_us":76,"threads_started":1}
I20260809 16:20:29.580771 24411 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60128:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 16:20:29.580992 24411 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 16:20:29.582494 24613 tablet_service.cc:1467] Processing CreateTablet for tablet ba78593c034148dd8151efb35b969fa8 (DEFAULT_TABLE table=test-table [id=2c1d5c852df9461289713fb9dfba044f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:20:29.582492 24835 tablet_service.cc:1467] Processing CreateTablet for tablet ba78593c034148dd8151efb35b969fa8 (DEFAULT_TABLE table=test-table [id=2c1d5c852df9461289713fb9dfba044f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:20:29.582536 25060 tablet_service.cc:1467] Processing CreateTablet for tablet ba78593c034148dd8151efb35b969fa8 (DEFAULT_TABLE table=test-table [id=2c1d5c852df9461289713fb9dfba044f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:20:29.582640 24835 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ba78593c034148dd8151efb35b969fa8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:20:29.582643 24613 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ba78593c034148dd8151efb35b969fa8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:20:29.582842 25060 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ba78593c034148dd8151efb35b969fa8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:20:29.584038 25340 tablet_bootstrap.cc:492] T ba78593c034148dd8151efb35b969fa8 P b40e1c9e9a4640108769f458b51d8690: Bootstrap starting.
I20260809 16:20:29.584285 25344 tablet_bootstrap.cc:492] T ba78593c034148dd8151efb35b969fa8 P 18fb7927ff43460790088c7b6465d2fb: Bootstrap starting.
I20260809 16:20:29.584333 25343 tablet_bootstrap.cc:492] T ba78593c034148dd8151efb35b969fa8 P 4f1796cdf8d4421fb7f9d0f614743bdc: Bootstrap starting.
I20260809 16:20:29.584447 25340 tablet_bootstrap.cc:654] T ba78593c034148dd8151efb35b969fa8 P b40e1c9e9a4640108769f458b51d8690: Neither blocks nor log segments found. Creating new log.
I20260809 16:20:29.584733 25344 tablet_bootstrap.cc:654] T ba78593c034148dd8151efb35b969fa8 P 18fb7927ff43460790088c7b6465d2fb: Neither blocks nor log segments found. Creating new log.
I20260809 16:20:29.584842 25343 tablet_bootstrap.cc:654] T ba78593c034148dd8151efb35b969fa8 P 4f1796cdf8d4421fb7f9d0f614743bdc: Neither blocks nor log segments found. Creating new log.
I20260809 16:20:29.585129 25340 tablet_bootstrap.cc:492] T ba78593c034148dd8151efb35b969fa8 P b40e1c9e9a4640108769f458b51d8690: No bootstrap required, opened a new log
I20260809 16:20:29.585220 25340 ts_tablet_manager.cc:1397] T ba78593c034148dd8151efb35b969fa8 P b40e1c9e9a4640108769f458b51d8690: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260809 16:20:29.585364 25344 tablet_bootstrap.cc:492] T ba78593c034148dd8151efb35b969fa8 P 18fb7927ff43460790088c7b6465d2fb: No bootstrap required, opened a new log
I20260809 16:20:29.585434 25344 ts_tablet_manager.cc:1397] T ba78593c034148dd8151efb35b969fa8 P 18fb7927ff43460790088c7b6465d2fb: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260809 16:20:29.585382 25340 raft_consensus.cc:348] T ba78593c034148dd8151efb35b969fa8 P b40e1c9e9a4640108769f458b51d8690 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1796cdf8d4421fb7f9d0f614743bdc" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 39087 } } peers { permanent_uuid: "18fb7927ff43460790088c7b6465d2fb" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 43831 } } peers { permanent_uuid: "b40e1c9e9a4640108769f458b51d8690" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 45353 } }
I20260809 16:20:29.585479 25340 raft_consensus.cc:374] T ba78593c034148dd8151efb35b969fa8 P b40e1c9e9a4640108769f458b51d8690 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:20:29.585510 25340 raft_consensus.cc:729] T ba78593c034148dd8151efb35b969fa8 P b40e1c9e9a4640108769f458b51d8690 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b40e1c9e9a4640108769f458b51d8690, State: Initialized, Role: FOLLOWER
I20260809 16:20:29.585561 25344 raft_consensus.cc:348] T ba78593c034148dd8151efb35b969fa8 P 18fb7927ff43460790088c7b6465d2fb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1796cdf8d4421fb7f9d0f614743bdc" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 39087 } } peers { permanent_uuid: "18fb7927ff43460790088c7b6465d2fb" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 43831 } } peers { permanent_uuid: "b40e1c9e9a4640108769f458b51d8690" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 45353 } }
I20260809 16:20:29.585589 25340 consensus_queue.cc:260] T ba78593c034148dd8151efb35b969fa8 P b40e1c9e9a4640108769f458b51d8690 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1796cdf8d4421fb7f9d0f614743bdc" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 39087 } } peers { permanent_uuid: "18fb7927ff43460790088c7b6465d2fb" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 43831 } } peers { permanent_uuid: "b40e1c9e9a4640108769f458b51d8690" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 45353 } }
I20260809 16:20:29.585650 25344 raft_consensus.cc:374] T ba78593c034148dd8151efb35b969fa8 P 18fb7927ff43460790088c7b6465d2fb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:20:29.585721 25340 ts_tablet_manager.cc:1428] T ba78593c034148dd8151efb35b969fa8 P b40e1c9e9a4640108769f458b51d8690: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 16:20:29.585798 25344 raft_consensus.cc:729] T ba78593c034148dd8151efb35b969fa8 P 18fb7927ff43460790088c7b6465d2fb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 18fb7927ff43460790088c7b6465d2fb, State: Initialized, Role: FOLLOWER
I20260809 16:20:29.585871 25344 consensus_queue.cc:260] T ba78593c034148dd8151efb35b969fa8 P 18fb7927ff43460790088c7b6465d2fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1796cdf8d4421fb7f9d0f614743bdc" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 39087 } } peers { permanent_uuid: "18fb7927ff43460790088c7b6465d2fb" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 43831 } } peers { permanent_uuid: "b40e1c9e9a4640108769f458b51d8690" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 45353 } }
I20260809 16:20:29.585981 25344 ts_tablet_manager.cc:1428] T ba78593c034148dd8151efb35b969fa8 P 18fb7927ff43460790088c7b6465d2fb: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260809 16:20:29.586575 25343 tablet_bootstrap.cc:492] T ba78593c034148dd8151efb35b969fa8 P 4f1796cdf8d4421fb7f9d0f614743bdc: No bootstrap required, opened a new log
I20260809 16:20:29.586683 25343 ts_tablet_manager.cc:1397] T ba78593c034148dd8151efb35b969fa8 P 4f1796cdf8d4421fb7f9d0f614743bdc: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260809 16:20:29.586869 25343 raft_consensus.cc:348] T ba78593c034148dd8151efb35b969fa8 P 4f1796cdf8d4421fb7f9d0f614743bdc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1796cdf8d4421fb7f9d0f614743bdc" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 39087 } } peers { permanent_uuid: "18fb7927ff43460790088c7b6465d2fb" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 43831 } } peers { permanent_uuid: "b40e1c9e9a4640108769f458b51d8690" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 45353 } }
I20260809 16:20:29.586956 25343 raft_consensus.cc:374] T ba78593c034148dd8151efb35b969fa8 P 4f1796cdf8d4421fb7f9d0f614743bdc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:20:29.586993 25343 raft_consensus.cc:729] T ba78593c034148dd8151efb35b969fa8 P 4f1796cdf8d4421fb7f9d0f614743bdc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f1796cdf8d4421fb7f9d0f614743bdc, State: Initialized, Role: FOLLOWER
I20260809 16:20:29.587055 25343 consensus_queue.cc:260] T ba78593c034148dd8151efb35b969fa8 P 4f1796cdf8d4421fb7f9d0f614743bdc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1796cdf8d4421fb7f9d0f614743bdc" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 39087 } } peers { permanent_uuid: "18fb7927ff43460790088c7b6465d2fb" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 43831 } } peers { permanent_uuid: "b40e1c9e9a4640108769f458b51d8690" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 45353 } }
I20260809 16:20:29.587255 25343 ts_tablet_manager.cc:1428] T ba78593c034148dd8151efb35b969fa8 P 4f1796cdf8d4421fb7f9d0f614743bdc: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260809 16:20:29.638892 25312 raft_consensus.cc:482] T ba78593c034148dd8151efb35b969fa8 P b40e1c9e9a4640108769f458b51d8690 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 16:20:29.638998 25312 raft_consensus.cc:504] T ba78593c034148dd8151efb35b969fa8 P b40e1c9e9a4640108769f458b51d8690 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1796cdf8d4421fb7f9d0f614743bdc" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 39087 } } peers { permanent_uuid: "18fb7927ff43460790088c7b6465d2fb" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 43831 } } peers { permanent_uuid: "b40e1c9e9a4640108769f458b51d8690" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 45353 } }
I20260809 16:20:29.639202 25312 leader_election.cc:288] T ba78593c034148dd8151efb35b969fa8 P b40e1c9e9a4640108769f458b51d8690 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4f1796cdf8d4421fb7f9d0f614743bdc (127.21.51.129:39087), 18fb7927ff43460790088c7b6465d2fb (127.21.51.130:43831)
I20260809 16:20:29.639441 24878 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ba78593c034148dd8151efb35b969fa8" candidate_uuid: "b40e1c9e9a4640108769f458b51d8690" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "18fb7927ff43460790088c7b6465d2fb" is_pre_election: true
I20260809 16:20:29.639475 24647 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ba78593c034148dd8151efb35b969fa8" candidate_uuid: "b40e1c9e9a4640108769f458b51d8690" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f1796cdf8d4421fb7f9d0f614743bdc" is_pre_election: true
I20260809 16:20:29.639588 24878 raft_consensus.cc:2454] T ba78593c034148dd8151efb35b969fa8 P 18fb7927ff43460790088c7b6465d2fb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b40e1c9e9a4640108769f458b51d8690 in term 0.
I20260809 16:20:29.639588 24647 raft_consensus.cc:2454] T ba78593c034148dd8151efb35b969fa8 P 4f1796cdf8d4421fb7f9d0f614743bdc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b40e1c9e9a4640108769f458b51d8690 in term 0.
I20260809 16:20:29.639770 24987 leader_election.cc:302] T ba78593c034148dd8151efb35b969fa8 P b40e1c9e9a4640108769f458b51d8690 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 18fb7927ff43460790088c7b6465d2fb, b40e1c9e9a4640108769f458b51d8690; no voters:
I20260809 16:20:29.639914 25312 raft_consensus.cc:2781] T ba78593c034148dd8151efb35b969fa8 P b40e1c9e9a4640108769f458b51d8690 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 16:20:29.639986 25312 raft_consensus.cc:482] T ba78593c034148dd8151efb35b969fa8 P b40e1c9e9a4640108769f458b51d8690 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 16:20:29.640002 25312 raft_consensus.cc:3037] T ba78593c034148dd8151efb35b969fa8 P b40e1c9e9a4640108769f458b51d8690 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:20:29.640834 25312 raft_consensus.cc:504] T ba78593c034148dd8151efb35b969fa8 P b40e1c9e9a4640108769f458b51d8690 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1796cdf8d4421fb7f9d0f614743bdc" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 39087 } } peers { permanent_uuid: "18fb7927ff43460790088c7b6465d2fb" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 43831 } } peers { permanent_uuid: "b40e1c9e9a4640108769f458b51d8690" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 45353 } }
I20260809 16:20:29.640982 25312 leader_election.cc:288] T ba78593c034148dd8151efb35b969fa8 P b40e1c9e9a4640108769f458b51d8690 [CANDIDATE]: Term 1 election: Requested vote from peers 4f1796cdf8d4421fb7f9d0f614743bdc (127.21.51.129:39087), 18fb7927ff43460790088c7b6465d2fb (127.21.51.130:43831)
I20260809 16:20:29.641106 24647 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ba78593c034148dd8151efb35b969fa8" candidate_uuid: "b40e1c9e9a4640108769f458b51d8690" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f1796cdf8d4421fb7f9d0f614743bdc"
I20260809 16:20:29.641183 24647 raft_consensus.cc:3037] T ba78593c034148dd8151efb35b969fa8 P 4f1796cdf8d4421fb7f9d0f614743bdc [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:20:29.641149 24878 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ba78593c034148dd8151efb35b969fa8" candidate_uuid: "b40e1c9e9a4640108769f458b51d8690" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "18fb7927ff43460790088c7b6465d2fb"
I20260809 16:20:29.641222 24878 raft_consensus.cc:3037] T ba78593c034148dd8151efb35b969fa8 P 18fb7927ff43460790088c7b6465d2fb [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:20:29.641940 24647 raft_consensus.cc:2454] T ba78593c034148dd8151efb35b969fa8 P 4f1796cdf8d4421fb7f9d0f614743bdc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b40e1c9e9a4640108769f458b51d8690 in term 1.
I20260809 16:20:29.642032 24878 raft_consensus.cc:2454] T ba78593c034148dd8151efb35b969fa8 P 18fb7927ff43460790088c7b6465d2fb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b40e1c9e9a4640108769f458b51d8690 in term 1.
I20260809 16:20:29.642153 24986 leader_election.cc:302] T ba78593c034148dd8151efb35b969fa8 P b40e1c9e9a4640108769f458b51d8690 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1796cdf8d4421fb7f9d0f614743bdc, b40e1c9e9a4640108769f458b51d8690; no voters:
I20260809 16:20:29.642246 25312 raft_consensus.cc:2781] T ba78593c034148dd8151efb35b969fa8 P b40e1c9e9a4640108769f458b51d8690 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:20:29.642298 25312 raft_consensus.cc:686] T ba78593c034148dd8151efb35b969fa8 P b40e1c9e9a4640108769f458b51d8690 [term 1 LEADER]: Becoming Leader. State: Replica: b40e1c9e9a4640108769f458b51d8690, State: Running, Role: LEADER
I20260809 16:20:29.642349 25312 consensus_queue.cc:237] T ba78593c034148dd8151efb35b969fa8 P b40e1c9e9a4640108769f458b51d8690 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1796cdf8d4421fb7f9d0f614743bdc" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 39087 } } peers { permanent_uuid: "18fb7927ff43460790088c7b6465d2fb" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 43831 } } peers { permanent_uuid: "b40e1c9e9a4640108769f458b51d8690" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 45353 } }
I20260809 16:20:29.643031 24411 catalog_manager.cc:5495] T ba78593c034148dd8151efb35b969fa8 P b40e1c9e9a4640108769f458b51d8690 reported cstate change: term changed from 0 to 1, leader changed from <none> to b40e1c9e9a4640108769f458b51d8690 (127.21.51.131). New cstate: current_term: 1 leader_uuid: "b40e1c9e9a4640108769f458b51d8690" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1796cdf8d4421fb7f9d0f614743bdc" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 39087 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "18fb7927ff43460790088c7b6465d2fb" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 43831 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b40e1c9e9a4640108769f458b51d8690" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 45353 } health_report { overall_health: HEALTHY } } }
I20260809 16:20:29.661918 25354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:29.659437 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35414 (local address 127.21.51.131:45353)
0809 16:20:29.659586 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0809 16:20:29.659589 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 16:20:29.659596 (+ 7us) server_negotiation.cc:407] Connection header received
0809 16:20:29.659615 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:20:29.659615 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:20:29.659625 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:20:29.659687 (+ 62us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 16:20:29.659807 (+ 120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:29.660298 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:29.661197 (+ 899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:29.661420 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:29.661491 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:29.661642 (+ 151us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 16:20:29.661681 (+ 39us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 16:20:29.661755 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0809 16:20:29.661808 (+ 53us) server_negotiation.cc:299] Negotiation successful
0809 16:20:29.661831 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":116,"thread_start_us":74,"threads_started":1}
I20260809 16:20:29.664188 24878 raft_consensus.cc:1261] T ba78593c034148dd8151efb35b969fa8 P 18fb7927ff43460790088c7b6465d2fb [term 1 FOLLOWER]: Refusing update from remote peer b40e1c9e9a4640108769f458b51d8690: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 16:20:29.664361 24647 raft_consensus.cc:1261] T ba78593c034148dd8151efb35b969fa8 P 4f1796cdf8d4421fb7f9d0f614743bdc [term 1 FOLLOWER]: Refusing update from remote peer b40e1c9e9a4640108769f458b51d8690: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 16:20:29.664426 25317 consensus_queue.cc:1035] T ba78593c034148dd8151efb35b969fa8 P b40e1c9e9a4640108769f458b51d8690 [LEADER]: Connected to new peer: Peer: permanent_uuid: "18fb7927ff43460790088c7b6465d2fb" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 43831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 16:20:29.664575 25312 consensus_queue.cc:1035] T ba78593c034148dd8151efb35b969fa8 P b40e1c9e9a4640108769f458b51d8690 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f1796cdf8d4421fb7f9d0f614743bdc" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 39087 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 16:20:29.673676 24409 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60128:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 16:20:29.673761 24409 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60128:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 16:20:29.674355 24409 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4d1cbb0203184af5a564b1a0deb4dd3f: Sending DeleteTablet for 3 replicas of tablet ba78593c034148dd8151efb35b969fa8
I20260809 16:20:29.674567 24613 tablet_service.cc:1514] Processing DeleteTablet for tablet ba78593c034148dd8151efb35b969fa8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:20:29 UTC) from {username='kudu', principal='kudu/127.21.51.190@KRBTEST.COM'} at 127.0.0.1:44898
I20260809 16:20:29.674665 25326 tablet_replica.cc:330] T ba78593c034148dd8151efb35b969fa8 P 4f1796cdf8d4421fb7f9d0f614743bdc: stopping tablet replica
I20260809 16:20:29.674645 25060 tablet_service.cc:1514] Processing DeleteTablet for tablet ba78593c034148dd8151efb35b969fa8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:20:29 UTC) from {username='kudu', principal='kudu/127.21.51.190@KRBTEST.COM'} at 127.0.0.1:35402
I20260809 16:20:29.674687 24835 tablet_service.cc:1514] Processing DeleteTablet for tablet ba78593c034148dd8151efb35b969fa8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:20:29 UTC) from {username='kudu', principal='kudu/127.21.51.190@KRBTEST.COM'} at 127.0.0.1:39228
I20260809 16:20:29.674723 25326 raft_consensus.cc:2229] T ba78593c034148dd8151efb35b969fa8 P 4f1796cdf8d4421fb7f9d0f614743bdc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 16:20:29.674760 25328 tablet_replica.cc:330] T ba78593c034148dd8151efb35b969fa8 P 18fb7927ff43460790088c7b6465d2fb: stopping tablet replica
I20260809 16:20:29.674765 25326 raft_consensus.cc:2258] T ba78593c034148dd8151efb35b969fa8 P 4f1796cdf8d4421fb7f9d0f614743bdc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 16:20:29.674810 25328 raft_consensus.cc:2229] T ba78593c034148dd8151efb35b969fa8 P 18fb7927ff43460790088c7b6465d2fb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 16:20:29.674759 24409 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60128:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 16:20:29.674849 25328 raft_consensus.cc:2258] T ba78593c034148dd8151efb35b969fa8 P 18fb7927ff43460790088c7b6465d2fb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 16:20:29.674844 25325 tablet_replica.cc:330] T ba78593c034148dd8151efb35b969fa8 P b40e1c9e9a4640108769f458b51d8690: stopping tablet replica
W20260809 16:20:29.674893 24409 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 16:20:29.674896 25325 raft_consensus.cc:2229] T ba78593c034148dd8151efb35b969fa8 P b40e1c9e9a4640108769f458b51d8690 [term 1 LEADER]: Raft consensus shutting down.
I20260809 16:20:29.674944 25325 raft_consensus.cc:2258] T ba78593c034148dd8151efb35b969fa8 P b40e1c9e9a4640108769f458b51d8690 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 16:20:29.675154 25328 ts_tablet_manager.cc:1905] T ba78593c034148dd8151efb35b969fa8 P 18fb7927ff43460790088c7b6465d2fb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 16:20:29.675493 25326 ts_tablet_manager.cc:1905] T ba78593c034148dd8151efb35b969fa8 P 4f1796cdf8d4421fb7f9d0f614743bdc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 16:20:29.676246 24613 tablet_service.cc:1467] Processing CreateTablet for tablet b48b5981612b49c5ac99a9c5a881c97b (DEFAULT_TABLE table=test-table [id=ddca71676ac1422a82bcbd50708b3a56]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:20:29.676365 24835 tablet_service.cc:1467] Processing CreateTablet for tablet b48b5981612b49c5ac99a9c5a881c97b (DEFAULT_TABLE table=test-table [id=ddca71676ac1422a82bcbd50708b3a56]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:20:29.676434 24613 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b48b5981612b49c5ac99a9c5a881c97b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:20:29.676467 24835 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b48b5981612b49c5ac99a9c5a881c97b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:20:29.676879 25328 ts_tablet_manager.cc:1918] T ba78593c034148dd8151efb35b969fa8 P 18fb7927ff43460790088c7b6465d2fb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 16:20:29.676926 25328 log.cc:1196] T ba78593c034148dd8151efb35b969fa8 P 18fb7927ff43460790088c7b6465d2fb: Deleting WAL directory at /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786292417722232-21710-0/minicluster-data/ts-1/wal/wals/ba78593c034148dd8151efb35b969fa8
I20260809 16:20:29.677155 25328 ts_tablet_manager.cc:1939] T ba78593c034148dd8151efb35b969fa8 P 18fb7927ff43460790088c7b6465d2fb: Deleting consensus metadata
I20260809 16:20:29.677161 25326 ts_tablet_manager.cc:1918] T ba78593c034148dd8151efb35b969fa8 P 4f1796cdf8d4421fb7f9d0f614743bdc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 16:20:29.677210 25326 log.cc:1196] T ba78593c034148dd8151efb35b969fa8 P 4f1796cdf8d4421fb7f9d0f614743bdc: Deleting WAL directory at /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786292417722232-21710-0/minicluster-data/ts-0/wal/wals/ba78593c034148dd8151efb35b969fa8
I20260809 16:20:29.677438 25326 ts_tablet_manager.cc:1939] T ba78593c034148dd8151efb35b969fa8 P 4f1796cdf8d4421fb7f9d0f614743bdc: Deleting consensus metadata
I20260809 16:20:29.677700 25343 tablet_bootstrap.cc:492] T b48b5981612b49c5ac99a9c5a881c97b P 4f1796cdf8d4421fb7f9d0f614743bdc: Bootstrap starting.
I20260809 16:20:29.677778 24388 catalog_manager.cc:4841] TS 4f1796cdf8d4421fb7f9d0f614743bdc (127.21.51.129:39087): tablet ba78593c034148dd8151efb35b969fa8 (table test-table [id=2c1d5c852df9461289713fb9dfba044f]) successfully deleted
I20260809 16:20:29.677794 25344 tablet_bootstrap.cc:492] T b48b5981612b49c5ac99a9c5a881c97b P 18fb7927ff43460790088c7b6465d2fb: Bootstrap starting.
I20260809 16:20:29.677791 25325 ts_tablet_manager.cc:1905] T ba78593c034148dd8151efb35b969fa8 P b40e1c9e9a4640108769f458b51d8690: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 16:20:29.678076 25344 tablet_bootstrap.cc:654] T b48b5981612b49c5ac99a9c5a881c97b P 18fb7927ff43460790088c7b6465d2fb: Neither blocks nor log segments found. Creating new log.
I20260809 16:20:29.678354 25343 tablet_bootstrap.cc:654] T b48b5981612b49c5ac99a9c5a881c97b P 4f1796cdf8d4421fb7f9d0f614743bdc: Neither blocks nor log segments found. Creating new log.
I20260809 16:20:29.678411 25060 tablet_service.cc:1467] Processing CreateTablet for tablet b48b5981612b49c5ac99a9c5a881c97b (DEFAULT_TABLE table=test-table [id=ddca71676ac1422a82bcbd50708b3a56]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:20:29.678507 25060 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b48b5981612b49c5ac99a9c5a881c97b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:20:29.679003 25343 tablet_bootstrap.cc:492] T b48b5981612b49c5ac99a9c5a881c97b P 4f1796cdf8d4421fb7f9d0f614743bdc: No bootstrap required, opened a new log
I20260809 16:20:29.679059 25343 ts_tablet_manager.cc:1397] T b48b5981612b49c5ac99a9c5a881c97b P 4f1796cdf8d4421fb7f9d0f614743bdc: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260809 16:20:29.679206 25343 raft_consensus.cc:348] T b48b5981612b49c5ac99a9c5a881c97b P 4f1796cdf8d4421fb7f9d0f614743bdc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1796cdf8d4421fb7f9d0f614743bdc" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 39087 } } peers { permanent_uuid: "18fb7927ff43460790088c7b6465d2fb" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 43831 } } peers { permanent_uuid: "b40e1c9e9a4640108769f458b51d8690" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 45353 } }
I20260809 16:20:29.679283 25343 raft_consensus.cc:374] T b48b5981612b49c5ac99a9c5a881c97b P 4f1796cdf8d4421fb7f9d0f614743bdc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:20:29.679306 25343 raft_consensus.cc:729] T b48b5981612b49c5ac99a9c5a881c97b P 4f1796cdf8d4421fb7f9d0f614743bdc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f1796cdf8d4421fb7f9d0f614743bdc, State: Initialized, Role: FOLLOWER
I20260809 16:20:29.679347 25343 consensus_queue.cc:260] T b48b5981612b49c5ac99a9c5a881c97b P 4f1796cdf8d4421fb7f9d0f614743bdc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1796cdf8d4421fb7f9d0f614743bdc" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 39087 } } peers { permanent_uuid: "18fb7927ff43460790088c7b6465d2fb" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 43831 } } peers { permanent_uuid: "b40e1c9e9a4640108769f458b51d8690" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 45353 } }
I20260809 16:20:29.679448 25343 ts_tablet_manager.cc:1428] T b48b5981612b49c5ac99a9c5a881c97b P 4f1796cdf8d4421fb7f9d0f614743bdc: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260809 16:20:29.679533 25325 ts_tablet_manager.cc:1918] T ba78593c034148dd8151efb35b969fa8 P b40e1c9e9a4640108769f458b51d8690: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 16:20:29.679593 25325 log.cc:1196] T ba78593c034148dd8151efb35b969fa8 P b40e1c9e9a4640108769f458b51d8690: Deleting WAL directory at /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786292417722232-21710-0/minicluster-data/ts-2/wal/wals/ba78593c034148dd8151efb35b969fa8
I20260809 16:20:29.679822 25340 tablet_bootstrap.cc:492] T b48b5981612b49c5ac99a9c5a881c97b P b40e1c9e9a4640108769f458b51d8690: Bootstrap starting.
I20260809 16:20:29.680059 24389 catalog_manager.cc:4841] TS 18fb7927ff43460790088c7b6465d2fb (127.21.51.130:43831): tablet ba78593c034148dd8151efb35b969fa8 (table test-table [id=2c1d5c852df9461289713fb9dfba044f]) successfully deleted
I20260809 16:20:29.680374 25340 tablet_bootstrap.cc:654] T b48b5981612b49c5ac99a9c5a881c97b P b40e1c9e9a4640108769f458b51d8690: Neither blocks nor log segments found. Creating new log.
I20260809 16:20:29.680400 25344 tablet_bootstrap.cc:492] T b48b5981612b49c5ac99a9c5a881c97b P 18fb7927ff43460790088c7b6465d2fb: No bootstrap required, opened a new log
I20260809 16:20:29.680439 25344 ts_tablet_manager.cc:1397] T b48b5981612b49c5ac99a9c5a881c97b P 18fb7927ff43460790088c7b6465d2fb: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.000s
I20260809 16:20:29.680568 25344 raft_consensus.cc:348] T b48b5981612b49c5ac99a9c5a881c97b P 18fb7927ff43460790088c7b6465d2fb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1796cdf8d4421fb7f9d0f614743bdc" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 39087 } } peers { permanent_uuid: "18fb7927ff43460790088c7b6465d2fb" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 43831 } } peers { permanent_uuid: "b40e1c9e9a4640108769f458b51d8690" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 45353 } }
I20260809 16:20:29.679838 25325 ts_tablet_manager.cc:1939] T ba78593c034148dd8151efb35b969fa8 P b40e1c9e9a4640108769f458b51d8690: Deleting consensus metadata
I20260809 16:20:29.680629 25344 raft_consensus.cc:374] T b48b5981612b49c5ac99a9c5a881c97b P 18fb7927ff43460790088c7b6465d2fb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:20:29.680644 25344 raft_consensus.cc:729] T b48b5981612b49c5ac99a9c5a881c97b P 18fb7927ff43460790088c7b6465d2fb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 18fb7927ff43460790088c7b6465d2fb, State: Initialized, Role: FOLLOWER
I20260809 16:20:29.680716 25344 consensus_queue.cc:260] T b48b5981612b49c5ac99a9c5a881c97b P 18fb7927ff43460790088c7b6465d2fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1796cdf8d4421fb7f9d0f614743bdc" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 39087 } } peers { permanent_uuid: "18fb7927ff43460790088c7b6465d2fb" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 43831 } } peers { permanent_uuid: "b40e1c9e9a4640108769f458b51d8690" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 45353 } }
I20260809 16:20:29.680804 25344 ts_tablet_manager.cc:1428] T b48b5981612b49c5ac99a9c5a881c97b P 18fb7927ff43460790088c7b6465d2fb: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 16:20:29.680984 24389 catalog_manager.cc:4841] TS b40e1c9e9a4640108769f458b51d8690 (127.21.51.131:45353): tablet ba78593c034148dd8151efb35b969fa8 (table test-table [id=2c1d5c852df9461289713fb9dfba044f]) successfully deleted
I20260809 16:20:29.681147 25340 tablet_bootstrap.cc:492] T b48b5981612b49c5ac99a9c5a881c97b P b40e1c9e9a4640108769f458b51d8690: No bootstrap required, opened a new log
I20260809 16:20:29.681196 25340 ts_tablet_manager.cc:1397] T b48b5981612b49c5ac99a9c5a881c97b P b40e1c9e9a4640108769f458b51d8690: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260809 16:20:29.681298 25340 raft_consensus.cc:348] T b48b5981612b49c5ac99a9c5a881c97b P b40e1c9e9a4640108769f458b51d8690 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1796cdf8d4421fb7f9d0f614743bdc" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 39087 } } peers { permanent_uuid: "18fb7927ff43460790088c7b6465d2fb" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 43831 } } peers { permanent_uuid: "b40e1c9e9a4640108769f458b51d8690" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 45353 } }
I20260809 16:20:29.681358 25340 raft_consensus.cc:374] T b48b5981612b49c5ac99a9c5a881c97b P b40e1c9e9a4640108769f458b51d8690 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:20:29.681373 25340 raft_consensus.cc:729] T b48b5981612b49c5ac99a9c5a881c97b P b40e1c9e9a4640108769f458b51d8690 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b40e1c9e9a4640108769f458b51d8690, State: Initialized, Role: FOLLOWER
I20260809 16:20:29.681404 25340 consensus_queue.cc:260] T b48b5981612b49c5ac99a9c5a881c97b P b40e1c9e9a4640108769f458b51d8690 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1796cdf8d4421fb7f9d0f614743bdc" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 39087 } } peers { permanent_uuid: "18fb7927ff43460790088c7b6465d2fb" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 43831 } } peers { permanent_uuid: "b40e1c9e9a4640108769f458b51d8690" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 45353 } }
I20260809 16:20:29.681495 25340 ts_tablet_manager.cc:1428] T b48b5981612b49c5ac99a9c5a881c97b P b40e1c9e9a4640108769f458b51d8690: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 16:20:29.751894 25351 raft_consensus.cc:482] T b48b5981612b49c5ac99a9c5a881c97b P 4f1796cdf8d4421fb7f9d0f614743bdc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 16:20:29.751996 25351 raft_consensus.cc:504] T b48b5981612b49c5ac99a9c5a881c97b P 4f1796cdf8d4421fb7f9d0f614743bdc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1796cdf8d4421fb7f9d0f614743bdc" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 39087 } } peers { permanent_uuid: "18fb7927ff43460790088c7b6465d2fb" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 43831 } } peers { permanent_uuid: "b40e1c9e9a4640108769f458b51d8690" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 45353 } }
I20260809 16:20:29.752375 25351 leader_election.cc:288] T b48b5981612b49c5ac99a9c5a881c97b P 4f1796cdf8d4421fb7f9d0f614743bdc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 18fb7927ff43460790088c7b6465d2fb (127.21.51.130:43831), b40e1c9e9a4640108769f458b51d8690 (127.21.51.131:45353)
I20260809 16:20:29.756737 25354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:29.752930 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.51.129:37355 (local address 127.21.51.131:45353)
0809 16:20:29.752979 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0809 16:20:29.752981 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:20:29.753015 (+ 34us) server_negotiation.cc:407] Connection header received
0809 16:20:29.753074 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:20:29.753075 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:20:29.753084 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:20:29.753152 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:20:29.753300 (+ 148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:29.753997 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:29.755730 (+ 1733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:29.756293 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:29.756376 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:29.756389 (+ 13us) server_negotiation.cc:1035] Waiting for connection context
0809 16:20:29.756579 (+ 190us) server_negotiation.cc:299] Negotiation successful
0809 16:20:29.756634 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260809 16:20:29.756748 25365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:29.752796 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.131:45353 (local address 127.21.51.129:37355)
0809 16:20:29.752953 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0809 16:20:29.752963 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 16:20:29.753018 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:20:29.753154 (+ 136us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:20:29.753155 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:20:29.753159 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:20:29.753230 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:20:29.753231 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:20:29.754103 (+ 872us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:20:29.754105 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:20:29.755589 (+ 1484us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:20:29.755593 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:20:29.756396 (+ 803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:20:29.756398 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:20:29.756481 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:29.756487 (+ 6us) client_negotiation.cc:769] Sending connection context
0809 16:20:29.756550 (+ 63us) client_negotiation.cc:240] Negotiation successful
0809 16:20:29.756593 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"mutex_wait_us":48,"thread_start_us":115,"threads_started":1}
I20260809 16:20:29.756973 25364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:29.752645 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.51.129:49243 (local address 127.21.51.130:43831)
0809 16:20:29.752796 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0809 16:20:29.752798 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:20:29.752812 (+ 14us) server_negotiation.cc:407] Connection header received
0809 16:20:29.752845 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:20:29.752846 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:20:29.752858 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:20:29.752968 (+ 110us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:20:29.753154 (+ 186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:29.754000 (+ 846us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:29.755761 (+ 1761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:29.756330 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:29.756398 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:29.756407 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0809 16:20:29.756845 (+ 438us) server_negotiation.cc:299] Negotiation successful
0809 16:20:29.756905 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":100,"threads_started":1}
I20260809 16:20:29.756783 25362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:29.752551 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.130:43831 (local address 127.21.51.129:49243)
0809 16:20:29.752720 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0809 16:20:29.752734 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 16:20:29.752789 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:20:29.752990 (+ 201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:20:29.752991 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:20:29.752993 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:20:29.753078 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:20:29.753080 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:20:29.754126 (+ 1046us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:20:29.754127 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:20:29.755660 (+ 1533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:20:29.755664 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:20:29.756447 (+ 783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:20:29.756448 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:20:29.756548 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:29.756553 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 16:20:29.756609 (+ 56us) client_negotiation.cc:240] Negotiation successful
0809 16:20:29.756648 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149,"mutex_wait_us":137,"thread_start_us":98,"threads_started":1}
I20260809 16:20:29.757081 25091 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b48b5981612b49c5ac99a9c5a881c97b" candidate_uuid: "4f1796cdf8d4421fb7f9d0f614743bdc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b40e1c9e9a4640108769f458b51d8690" is_pre_election: true
I20260809 16:20:29.757201 25091 raft_consensus.cc:2454] T b48b5981612b49c5ac99a9c5a881c97b P b40e1c9e9a4640108769f458b51d8690 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1796cdf8d4421fb7f9d0f614743bdc in term 0.
I20260809 16:20:29.757330 24878 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b48b5981612b49c5ac99a9c5a881c97b" candidate_uuid: "4f1796cdf8d4421fb7f9d0f614743bdc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "18fb7927ff43460790088c7b6465d2fb" is_pre_election: true
I20260809 16:20:29.757436 24878 raft_consensus.cc:2454] T b48b5981612b49c5ac99a9c5a881c97b P 18fb7927ff43460790088c7b6465d2fb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1796cdf8d4421fb7f9d0f614743bdc in term 0.
I20260809 16:20:29.757418 24539 leader_election.cc:302] T b48b5981612b49c5ac99a9c5a881c97b P 4f1796cdf8d4421fb7f9d0f614743bdc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1796cdf8d4421fb7f9d0f614743bdc, b40e1c9e9a4640108769f458b51d8690; no voters:
I20260809 16:20:29.757551 25351 raft_consensus.cc:2781] T b48b5981612b49c5ac99a9c5a881c97b P 4f1796cdf8d4421fb7f9d0f614743bdc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 16:20:29.757592 25351 raft_consensus.cc:482] T b48b5981612b49c5ac99a9c5a881c97b P 4f1796cdf8d4421fb7f9d0f614743bdc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 16:20:29.757611 25351 raft_consensus.cc:3037] T b48b5981612b49c5ac99a9c5a881c97b P 4f1796cdf8d4421fb7f9d0f614743bdc [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:20:29.758378 25351 raft_consensus.cc:504] T b48b5981612b49c5ac99a9c5a881c97b P 4f1796cdf8d4421fb7f9d0f614743bdc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1796cdf8d4421fb7f9d0f614743bdc" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 39087 } } peers { permanent_uuid: "18fb7927ff43460790088c7b6465d2fb" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 43831 } } peers { permanent_uuid: "b40e1c9e9a4640108769f458b51d8690" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 45353 } }
I20260809 16:20:29.758512 25351 leader_election.cc:288] T b48b5981612b49c5ac99a9c5a881c97b P 4f1796cdf8d4421fb7f9d0f614743bdc [CANDIDATE]: Term 1 election: Requested vote from peers 18fb7927ff43460790088c7b6465d2fb (127.21.51.130:43831), b40e1c9e9a4640108769f458b51d8690 (127.21.51.131:45353)
I20260809 16:20:29.758708 25091 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b48b5981612b49c5ac99a9c5a881c97b" candidate_uuid: "4f1796cdf8d4421fb7f9d0f614743bdc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b40e1c9e9a4640108769f458b51d8690"
I20260809 16:20:29.758790 25091 raft_consensus.cc:3037] T b48b5981612b49c5ac99a9c5a881c97b P b40e1c9e9a4640108769f458b51d8690 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:20:29.758733 24878 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b48b5981612b49c5ac99a9c5a881c97b" candidate_uuid: "4f1796cdf8d4421fb7f9d0f614743bdc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "18fb7927ff43460790088c7b6465d2fb"
I20260809 16:20:29.758824 24878 raft_consensus.cc:3037] T b48b5981612b49c5ac99a9c5a881c97b P 18fb7927ff43460790088c7b6465d2fb [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:20:29.759390 25091 raft_consensus.cc:2454] T b48b5981612b49c5ac99a9c5a881c97b P b40e1c9e9a4640108769f458b51d8690 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1796cdf8d4421fb7f9d0f614743bdc in term 1.
I20260809 16:20:29.759486 24878 raft_consensus.cc:2454] T b48b5981612b49c5ac99a9c5a881c97b P 18fb7927ff43460790088c7b6465d2fb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1796cdf8d4421fb7f9d0f614743bdc in term 1.
I20260809 16:20:29.759557 24539 leader_election.cc:302] T b48b5981612b49c5ac99a9c5a881c97b P 4f1796cdf8d4421fb7f9d0f614743bdc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1796cdf8d4421fb7f9d0f614743bdc, b40e1c9e9a4640108769f458b51d8690; no voters:
I20260809 16:20:29.759644 25351 raft_consensus.cc:2781] T b48b5981612b49c5ac99a9c5a881c97b P 4f1796cdf8d4421fb7f9d0f614743bdc [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:20:29.759799 25351 raft_consensus.cc:686] T b48b5981612b49c5ac99a9c5a881c97b P 4f1796cdf8d4421fb7f9d0f614743bdc [term 1 LEADER]: Becoming Leader. State: Replica: 4f1796cdf8d4421fb7f9d0f614743bdc, State: Running, Role: LEADER
I20260809 16:20:29.759910 25351 consensus_queue.cc:237] T b48b5981612b49c5ac99a9c5a881c97b P 4f1796cdf8d4421fb7f9d0f614743bdc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1796cdf8d4421fb7f9d0f614743bdc" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 39087 } } peers { permanent_uuid: "18fb7927ff43460790088c7b6465d2fb" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 43831 } } peers { permanent_uuid: "b40e1c9e9a4640108769f458b51d8690" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 45353 } }
I20260809 16:20:29.760612 24411 catalog_manager.cc:5495] T b48b5981612b49c5ac99a9c5a881c97b P 4f1796cdf8d4421fb7f9d0f614743bdc reported cstate change: term changed from 0 to 1, leader changed from <none> to 4f1796cdf8d4421fb7f9d0f614743bdc (127.21.51.129). New cstate: current_term: 1 leader_uuid: "4f1796cdf8d4421fb7f9d0f614743bdc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f1796cdf8d4421fb7f9d0f614743bdc" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 39087 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "18fb7927ff43460790088c7b6465d2fb" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 43831 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b40e1c9e9a4640108769f458b51d8690" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 45353 } health_report { overall_health: UNKNOWN } } }
I20260809 16:20:29.780587 25369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:29.778047 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44900 (local address 127.21.51.129:39087)
0809 16:20:29.778209 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0809 16:20:29.778211 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:20:29.778218 (+ 7us) server_negotiation.cc:407] Connection header received
0809 16:20:29.778260 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:20:29.778261 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:20:29.778279 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:20:29.778372 (+ 93us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 16:20:29.778479 (+ 107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:29.778992 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:29.779724 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:29.779968 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:29.780046 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:29.780209 (+ 163us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 16:20:29.780332 (+ 123us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 16:20:29.780422 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0809 16:20:29.780464 (+ 42us) server_negotiation.cc:299] Negotiation successful
0809 16:20:29.780483 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":139,"thread_start_us":84,"threads_started":1}
I20260809 16:20:29.785411 25364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:29.783057 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.51.129:56447 (local address 127.21.51.130:43831)
0809 16:20:29.783105 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0809 16:20:29.783107 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:20:29.783143 (+ 36us) server_negotiation.cc:407] Connection header received
0809 16:20:29.783208 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:20:29.783209 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:20:29.783220 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:20:29.783284 (+ 64us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 16:20:29.783423 (+ 139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:29.783875 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:29.784639 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:29.784882 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:29.784964 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:29.785163 (+ 199us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 16:20:29.785200 (+ 37us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 16:20:29.785264 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0809 16:20:29.785315 (+ 51us) server_negotiation.cc:299] Negotiation successful
0809 16:20:29.785337 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260809 16:20:29.785449 25374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:29.782916 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.130:43831 (local address 127.21.51.129:56447)
0809 16:20:29.783082 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0809 16:20:29.783095 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 16:20:29.783157 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:20:29.783291 (+ 134us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:20:29.783291 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:20:29.783293 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 16:20:29.783354 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:20:29.783355 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:20:29.783958 (+ 603us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:20:29.783959 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:20:29.784548 (+ 589us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:20:29.784550 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:20:29.784984 (+ 434us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:20:29.784986 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:20:29.785056 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:29.785069 (+ 13us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 16:20:29.785261 (+ 192us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 16:20:29.785263 (+ 2us) client_negotiation.cc:769] Sending connection context
0809 16:20:29.785312 (+ 49us) client_negotiation.cc:240] Negotiation successful
0809 16:20:29.785352 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":92,"threads_started":1}
I20260809 16:20:29.787429 25091 raft_consensus.cc:1261] T b48b5981612b49c5ac99a9c5a881c97b P b40e1c9e9a4640108769f458b51d8690 [term 1 FOLLOWER]: Refusing update from remote peer 4f1796cdf8d4421fb7f9d0f614743bdc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 16:20:29.787429 24878 raft_consensus.cc:1261] T b48b5981612b49c5ac99a9c5a881c97b P 18fb7927ff43460790088c7b6465d2fb [term 1 FOLLOWER]: Refusing update from remote peer 4f1796cdf8d4421fb7f9d0f614743bdc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 16:20:29.787710 25366 consensus_queue.cc:1035] T b48b5981612b49c5ac99a9c5a881c97b P 4f1796cdf8d4421fb7f9d0f614743bdc [LEADER]: Connected to new peer: Peer: permanent_uuid: "18fb7927ff43460790088c7b6465d2fb" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 43831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 16:20:29.787854 25351 consensus_queue.cc:1035] T b48b5981612b49c5ac99a9c5a881c97b P 4f1796cdf8d4421fb7f9d0f614743bdc [LEADER]: Connected to new peer: Peer: permanent_uuid: "b40e1c9e9a4640108769f458b51d8690" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 45353 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 16:20:29.795137 25369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:29.792898 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.51.130:47343 (local address 127.21.51.129:39087)
0809 16:20:29.792936 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0809 16:20:29.792937 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 16:20:29.793051 (+ 114us) server_negotiation.cc:407] Connection header received
0809 16:20:29.793112 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:20:29.793112 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:20:29.793123 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:20:29.793172 (+ 49us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 16:20:29.793270 (+ 98us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:29.793716 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:29.794444 (+ 728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:29.794698 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:29.794766 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:29.794924 (+ 158us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 16:20:29.794956 (+ 32us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 16:20:29.795004 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0809 16:20:29.795060 (+ 56us) server_negotiation.cc:299] Negotiation successful
0809 16:20:29.795078 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260809 16:20:29.795186 25380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:29.792861 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.129:39087 (local address 127.21.51.130:47343)
0809 16:20:29.792999 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0809 16:20:29.793010 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 16:20:29.793063 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:20:29.793173 (+ 110us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:20:29.793174 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:20:29.793176 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 16:20:29.793217 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:20:29.793218 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:20:29.793800 (+ 582us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:20:29.793800 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:20:29.794357 (+ 557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:20:29.794358 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:20:29.794766 (+ 408us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:20:29.794766 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:20:29.794821 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:29.794825 (+ 4us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 16:20:29.795003 (+ 178us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 16:20:29.795005 (+ 2us) client_negotiation.cc:769] Sending connection context
0809 16:20:29.795056 (+ 51us) client_negotiation.cc:240] Negotiation successful
0809 16:20:29.795103 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":91,"threads_started":1}
W20260809 16:20:29.795557 25300 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260809 16:20:31.319747 24409 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60128:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 16:20:31.319895 24409 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60128:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 16:20:31.320878 24409 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4d1cbb0203184af5a564b1a0deb4dd3f: Sending DeleteTablet for 3 replicas of tablet b48b5981612b49c5ac99a9c5a881c97b
I20260809 16:20:31.321166 24835 tablet_service.cc:1514] Processing DeleteTablet for tablet b48b5981612b49c5ac99a9c5a881c97b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:20:31 UTC) from {username='kudu', principal='kudu/127.21.51.190@KRBTEST.COM'} at 127.0.0.1:39228
I20260809 16:20:31.321188 25060 tablet_service.cc:1514] Processing DeleteTablet for tablet b48b5981612b49c5ac99a9c5a881c97b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:20:31 UTC) from {username='kudu', principal='kudu/127.21.51.190@KRBTEST.COM'} at 127.0.0.1:35402
I20260809 16:20:31.321245 24613 tablet_service.cc:1514] Processing DeleteTablet for tablet b48b5981612b49c5ac99a9c5a881c97b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:20:31 UTC) from {username='kudu', principal='kudu/127.21.51.190@KRBTEST.COM'} at 127.0.0.1:44898
I20260809 16:20:31.321364 21710 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrNqUCn/build/release/bin/kudu with pid 24518
I20260809 16:20:31.329368 25416 tablet_replica.cc:330] T b48b5981612b49c5ac99a9c5a881c97b P b40e1c9e9a4640108769f458b51d8690: stopping tablet replica
I20260809 16:20:31.329494 25416 raft_consensus.cc:2229] T b48b5981612b49c5ac99a9c5a881c97b P b40e1c9e9a4640108769f458b51d8690 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 16:20:31.329550 25416 raft_consensus.cc:2258] T b48b5981612b49c5ac99a9c5a881c97b P b40e1c9e9a4640108769f458b51d8690 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 16:20:31.330138 25416 ts_tablet_manager.cc:1905] T b48b5981612b49c5ac99a9c5a881c97b P b40e1c9e9a4640108769f458b51d8690: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 16:20:31.332312 25416 ts_tablet_manager.cc:1918] T b48b5981612b49c5ac99a9c5a881c97b P b40e1c9e9a4640108769f458b51d8690: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260809 16:20:31.332428 25416 log.cc:1196] T b48b5981612b49c5ac99a9c5a881c97b P b40e1c9e9a4640108769f458b51d8690: Deleting WAL directory at /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786292417722232-21710-0/minicluster-data/ts-2/wal/wals/b48b5981612b49c5ac99a9c5a881c97b
I20260809 16:20:31.332722 25416 ts_tablet_manager.cc:1939] T b48b5981612b49c5ac99a9c5a881c97b P b40e1c9e9a4640108769f458b51d8690: Deleting consensus metadata
I20260809 16:20:31.337504 24389 catalog_manager.cc:4841] TS b40e1c9e9a4640108769f458b51d8690 (127.21.51.131:45353): tablet b48b5981612b49c5ac99a9c5a881c97b (table test-table [id=ddca71676ac1422a82bcbd50708b3a56]) successfully deleted
I20260809 16:20:31.340342 25411 tablet_replica.cc:330] T b48b5981612b49c5ac99a9c5a881c97b P 18fb7927ff43460790088c7b6465d2fb: stopping tablet replica
I20260809 16:20:31.340440 25411 raft_consensus.cc:2229] T b48b5981612b49c5ac99a9c5a881c97b P 18fb7927ff43460790088c7b6465d2fb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 16:20:31.340485 25411 raft_consensus.cc:2258] T b48b5981612b49c5ac99a9c5a881c97b P 18fb7927ff43460790088c7b6465d2fb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 16:20:31.340813 25411 ts_tablet_manager.cc:1905] T b48b5981612b49c5ac99a9c5a881c97b P 18fb7927ff43460790088c7b6465d2fb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 16:20:31.342342 25411 ts_tablet_manager.cc:1918] T b48b5981612b49c5ac99a9c5a881c97b P 18fb7927ff43460790088c7b6465d2fb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260809 16:20:31.342442 25411 log.cc:1196] T b48b5981612b49c5ac99a9c5a881c97b P 18fb7927ff43460790088c7b6465d2fb: Deleting WAL directory at /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786292417722232-21710-0/minicluster-data/ts-1/wal/wals/b48b5981612b49c5ac99a9c5a881c97b
I20260809 16:20:31.342684 25411 ts_tablet_manager.cc:1939] T b48b5981612b49c5ac99a9c5a881c97b P 18fb7927ff43460790088c7b6465d2fb: Deleting consensus metadata
I20260809 16:20:31.343080 24389 catalog_manager.cc:4841] TS 18fb7927ff43460790088c7b6465d2fb (127.21.51.130:43831): tablet b48b5981612b49c5ac99a9c5a881c97b (table test-table [id=ddca71676ac1422a82bcbd50708b3a56]) successfully deleted
W20260809 16:20:31.346197 24388 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.21.51.129:39087 (error 108)
I20260809 16:20:31.346807 25419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:31.346599 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.129:39087 (local address 127.0.0.1:44912)
0809 16:20:31.346722 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0809 16:20:31.346774 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.51.129:39087: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":91,"thread_start_us":56,"threads_started":1}
W20260809 16:20:31.346959 24388 catalog_manager.cc:4584] TS 4f1796cdf8d4421fb7f9d0f614743bdc (127.21.51.129:39087): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet b48b5981612b49c5ac99a9c5a881c97b: Network error: Client connection negotiation failed: client connection to 127.21.51.129:39087: connect: Connection refused (error 111)
I20260809 16:20:31.347028 21710 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrNqUCn/build/release/bin/kudu with pid 24733
I20260809 16:20:31.359248 21710 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrNqUCn/build/release/bin/kudu with pid 24963
I20260809 16:20:31.369134 21710 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrNqUCn/build/release/bin/kudu with pid 24357
2026-08-09T16:20:31Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (6137 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786292417722232-21710-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 16:20:31 dist-test-slave-s11t krb5kdc[25424](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 16:20:31 dist-test-slave-s11t krb5kdc[25424](info): set up 2 sockets
Aug 09 16:20:31 dist-test-slave-s11t krb5kdc[25424](info): commencing operation
krb5kdc: starting...
W20260809 16:20:32.615741 21710 mini_kdc.cc:121] Time spent starting KDC: real 1.216s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786292417722232-21710-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786292417722232-21710-0/krb5kdc/test-user.keytab.
Aug 09 16:20:32 dist-test-slave-s11t krb5kdc[25424](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292432, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T16:20:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T16:20:32Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.51.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.51.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.51.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.190.keytab.
Entry for principal kudu/127.21.51.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.190.keytab.
I20260809 16:20:32.735241 21710 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrNqUCn/build/release/bin/kudu
/tmp/dist-test-taskrNqUCn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786292417722232-21710-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786292417722232-21710-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786292417722232-21710-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786292417722232-21710-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.51.190:33739
--webserver_interface=127.21.51.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.190.keytab
--principal=kudu/127.21.51.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.51.148:44071
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.51.190:33739
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786292417722232-21710-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786292417722232-21710-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786292417722232-21710-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:20:32.818145 25445 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:20:32.818341 25445 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:20:32.818363 25445 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:20:32.819545 25445 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 16:20:32.819584 25445 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:20:32.819598 25445 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:20:32.819612 25445 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 16:20:32.819624 25445 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 16:20:32.821107 25445 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.51.148:44071
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786292417722232-21710-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786292417722232-21710-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.51.190:33739
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.190.keytab
--principal=kudu/127.21.51.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.51.190:33739
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786292417722232-21710-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.51.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25445
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786292417722232-21710-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:20:32.821346 25445 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:20:32.821594 25445 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:20:32.826820 25451 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:20:32.826864 25454 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:20:32.826918 25452 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:20:32.826915 25445 server_base.cc:1034] running on GCE node
I20260809 16:20:32.827381 25445 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:20:32.827682 25445 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:20:32.828824 25445 hybrid_clock.cc:648] HybridClock initialized: now 1786292432828826 us; error 46 us; skew 500 ppm
Aug 09 16:20:32 dist-test-slave-s11t krb5kdc[25424](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292432, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.51.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:20:32.831403 25445 init.cc:377] Logged in from keytab as kudu/127.21.51.190@KRBTEST.COM (short username kudu)
I20260809 16:20:32.831883 25445 webserver.cc:460] Webserver started at http://127.21.51.190:33731/ using document root <none> and password file <none>
I20260809 16:20:32.832134 25445 fs_manager.cc:359] Metadata directory not provided
I20260809 16:20:32.832180 25445 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:20:32.832311 25445 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:20:32.833369 25445 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786292417722232-21710-0/minicluster-data/master-0/data/instance:
uuid: "171a0620ee7949f78593ac5ab7e4ce81"
format_stamp: "Formatted at 2026-08-09 16:20:32 on dist-test-slave-s11t"
server_key: "a767524e6e0d241b0b4341a81aa4636c"
server_key_iv: "40536e3d8f18e7be61be32b586a44256"
server_key_version: "encryptionkey@0"
I20260809 16:20:32.833726 25445 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786292417722232-21710-0/minicluster-data/master-0/wal/instance:
uuid: "171a0620ee7949f78593ac5ab7e4ce81"
format_stamp: "Formatted at 2026-08-09 16:20:32 on dist-test-slave-s11t"
server_key: "a767524e6e0d241b0b4341a81aa4636c"
server_key_iv: "40536e3d8f18e7be61be32b586a44256"
server_key_version: "encryptionkey@0"
I20260809 16:20:32.835119 25445 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260809 16:20:32.835852 25469 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:20:32.836052 25445 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 16:20:32.836161 25445 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786292417722232-21710-0/minicluster-data/master-0/data,/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786292417722232-21710-0/minicluster-data/master-0/wal
uuid: "171a0620ee7949f78593ac5ab7e4ce81"
format_stamp: "Formatted at 2026-08-09 16:20:32 on dist-test-slave-s11t"
server_key: "a767524e6e0d241b0b4341a81aa4636c"
server_key_iv: "40536e3d8f18e7be61be32b586a44256"
server_key_version: "encryptionkey@0"
I20260809 16:20:32.836236 25445 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786292417722232-21710-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786292417722232-21710-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786292417722232-21710-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:20:32.845662 25445 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:20:32.847826 25445 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:20:32.847993 25445 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:20:32.852116 25445 rpc_server.cc:307] RPC server started. Bound to: 127.21.51.190:33739
I20260809 16:20:32.852154 25548 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.51.190:33739 every 8 connection(s)
I20260809 16:20:32.852478 25445 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786292417722232-21710-0/minicluster-data/master-0/data/info.pb
I20260809 16:20:32.853354 25549 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:20:32.855465 25549 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 171a0620ee7949f78593ac5ab7e4ce81: Bootstrap starting.
I20260809 16:20:32.855971 25549 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 171a0620ee7949f78593ac5ab7e4ce81: Neither blocks nor log segments found. Creating new log.
I20260809 16:20:32.856201 25549 log.cc:824] T 00000000000000000000000000000000 P 171a0620ee7949f78593ac5ab7e4ce81: Log is configured to *not* fsync() on all Append() calls
I20260809 16:20:32.856866 25549 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 171a0620ee7949f78593ac5ab7e4ce81: No bootstrap required, opened a new log
I20260809 16:20:32.858381 25549 raft_consensus.cc:348] T 00000000000000000000000000000000 P 171a0620ee7949f78593ac5ab7e4ce81 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "171a0620ee7949f78593ac5ab7e4ce81" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 33739 } }
I20260809 16:20:32.858510 25549 raft_consensus.cc:374] T 00000000000000000000000000000000 P 171a0620ee7949f78593ac5ab7e4ce81 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:20:32.858532 25549 raft_consensus.cc:729] T 00000000000000000000000000000000 P 171a0620ee7949f78593ac5ab7e4ce81 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 171a0620ee7949f78593ac5ab7e4ce81, State: Initialized, Role: FOLLOWER
I20260809 16:20:32.858662 25549 consensus_queue.cc:260] T 00000000000000000000000000000000 P 171a0620ee7949f78593ac5ab7e4ce81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "171a0620ee7949f78593ac5ab7e4ce81" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 33739 } }
I20260809 16:20:32.858718 25549 raft_consensus.cc:388] T 00000000000000000000000000000000 P 171a0620ee7949f78593ac5ab7e4ce81 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 16:20:32.858739 25549 raft_consensus.cc:482] T 00000000000000000000000000000000 P 171a0620ee7949f78593ac5ab7e4ce81 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 16:20:32.858765 25549 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 171a0620ee7949f78593ac5ab7e4ce81 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:20:32.859242 25549 raft_consensus.cc:504] T 00000000000000000000000000000000 P 171a0620ee7949f78593ac5ab7e4ce81 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "171a0620ee7949f78593ac5ab7e4ce81" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 33739 } }
I20260809 16:20:32.859326 25549 leader_election.cc:302] T 00000000000000000000000000000000 P 171a0620ee7949f78593ac5ab7e4ce81 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 171a0620ee7949f78593ac5ab7e4ce81; no voters:
I20260809 16:20:32.859443 25549 leader_election.cc:288] T 00000000000000000000000000000000 P 171a0620ee7949f78593ac5ab7e4ce81 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 16:20:32.859511 25554 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 171a0620ee7949f78593ac5ab7e4ce81 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:20:32.859694 25554 raft_consensus.cc:686] T 00000000000000000000000000000000 P 171a0620ee7949f78593ac5ab7e4ce81 [term 1 LEADER]: Becoming Leader. State: Replica: 171a0620ee7949f78593ac5ab7e4ce81, State: Running, Role: LEADER
I20260809 16:20:32.859809 25549 sys_catalog.cc:564] T 00000000000000000000000000000000 P 171a0620ee7949f78593ac5ab7e4ce81 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 16:20:32.859902 25554 consensus_queue.cc:237] T 00000000000000000000000000000000 P 171a0620ee7949f78593ac5ab7e4ce81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "171a0620ee7949f78593ac5ab7e4ce81" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 33739 } }
I20260809 16:20:32.860014 21710 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrNqUCn/build/release/bin/kudu as pid 25445
I20260809 16:20:32.860103 21710 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786292417722232-21710-0/minicluster-data/master-0/wal/instance
I20260809 16:20:32.860296 21710 external_mini_cluster.cc:1428] Setting key 8d4d786444270e3121696b82308e4946
I20260809 16:20:32.860561 25557 sys_catalog.cc:455] T 00000000000000000000000000000000 P 171a0620ee7949f78593ac5ab7e4ce81 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 171a0620ee7949f78593ac5ab7e4ce81. Latest consensus state: current_term: 1 leader_uuid: "171a0620ee7949f78593ac5ab7e4ce81" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "171a0620ee7949f78593ac5ab7e4ce81" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 33739 } } }
I20260809 16:20:32.860649 25557 sys_catalog.cc:458] T 00000000000000000000000000000000 P 171a0620ee7949f78593ac5ab7e4ce81 [sys.catalog]: This master's current role is: LEADER
I20260809 16:20:32.860992 25556 sys_catalog.cc:455] T 00000000000000000000000000000000 P 171a0620ee7949f78593ac5ab7e4ce81 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "171a0620ee7949f78593ac5ab7e4ce81" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "171a0620ee7949f78593ac5ab7e4ce81" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 33739 } } }
I20260809 16:20:32.861068 25556 sys_catalog.cc:458] T 00000000000000000000000000000000 P 171a0620ee7949f78593ac5ab7e4ce81 [sys.catalog]: This master's current role is: LEADER
I20260809 16:20:32.861200 25563 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 16:20:32.861771 25563 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 16:20:32.863435 25563 catalog_manager.cc:1347] Generated new cluster ID: 2687a5fb79d247009b4da15e3053ca66
I20260809 16:20:32.863492 25563 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 09 16:20:32 dist-test-slave-s11t krb5kdc[25424](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292432, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.51.190@KRBTEST.COM
I20260809 16:20:32.870777 25571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:32.860987 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41464 (local address 127.21.51.190:33739)
0809 16:20:32.861160 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0809 16:20:32.861170 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 16:20:32.861202 (+ 32us) server_negotiation.cc:407] Connection header received
0809 16:20:32.861315 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:20:32.861321 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:20:32.861411 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:20:32.861594 (+ 183us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:20:32.862391 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:32.863743 (+ 1352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:32.864502 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:32.865059 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:32.865180 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:32.867454 (+ 2274us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:20:32.867457 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:20:32.867458 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:20:32.867469 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:20:32.869217 (+ 1748us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:20:32.869591 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:20:32.869594 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:20:32.869594 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:20:32.869695 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:20:32.869911 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:20:32.869912 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:20:32.869912 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:20:32.870139 (+ 227us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:20:32.870317 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0809 16:20:32.870385 (+ 68us) server_negotiation.cc:299] Negotiation successful
0809 16:20:32.870625 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":69,"threads_started":1}
I20260809 16:20:32.882284 25563 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 16:20:32.882846 25563 catalog_manager.cc:1504] Loading token signing keys...
I20260809 16:20:32.885579 25563 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 171a0620ee7949f78593ac5ab7e4ce81: Generated new TSK 0
I20260809 16:20:32.885787 25563 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.21.51.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.51.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.51.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.129.keytab.
Entry for principal kudu/127.21.51.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.129.keytab.
I20260809 16:20:32.911748 21710 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrNqUCn/build/release/bin/kudu
/tmp/dist-test-taskrNqUCn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786292417722232-21710-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786292417722232-21710-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786292417722232-21710-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786292417722232-21710-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.51.129:0
--local_ip_for_outbound_sockets=127.21.51.129
--webserver_interface=127.21.51.129
--webserver_port=0
--tserver_master_addrs=127.21.51.190:33739
--keytab_file=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.129.keytab
--principal=kudu/127.21.51.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.51.148:44071
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786292417722232-21710-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786292417722232-21710-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786292417722232-21710-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:20:32.994549 25589 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:20:32.994747 25589 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:20:32.994769 25589 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:20:32.995836 25589 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:20:32.995880 25589 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:20:32.995906 25589 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.51.129
I20260809 16:20:32.997421 25589 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.51.148:44071
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786292417722232-21710-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786292417722232-21710-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.129.keytab
--principal=kudu/127.21.51.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.51.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786292417722232-21710-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.51.129
--webserver_port=0
--tserver_master_addrs=127.21.51.190:33739
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25589
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.51.129
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786292417722232-21710-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:20:32.997609 25589 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:20:32.997788 25589 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:20:33.002790 25598 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:20:33.002831 25600 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:20:33.002887 25602 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:20:33.003002 25589 server_base.cc:1034] running on GCE node
I20260809 16:20:33.003232 25589 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:20:33.003465 25589 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:20:33.004657 25589 hybrid_clock.cc:648] HybridClock initialized: now 1786292433004632 us; error 35 us; skew 500 ppm
Aug 09 16:20:33 dist-test-slave-s11t krb5kdc[25424](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292433, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.51.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:20:33.007455 25589 init.cc:377] Logged in from keytab as kudu/127.21.51.129@KRBTEST.COM (short username kudu)
I20260809 16:20:33.007916 25589 webserver.cc:460] Webserver started at http://127.21.51.129:41485/ using document root <none> and password file <none>
I20260809 16:20:33.008118 25589 fs_manager.cc:359] Metadata directory not provided
I20260809 16:20:33.008160 25589 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:20:33.008251 25589 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:20:33.009326 25589 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786292417722232-21710-0/minicluster-data/ts-0/data/instance:
uuid: "9014e5969a5342d987cedce48d96c2b5"
format_stamp: "Formatted at 2026-08-09 16:20:33 on dist-test-slave-s11t"
server_key: "ba8ef8e230e3f285343f1a6dd4913961"
server_key_iv: "5b9fbab93d7c011607763ea5567d6313"
server_key_version: "encryptionkey@0"
I20260809 16:20:33.009639 25589 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786292417722232-21710-0/minicluster-data/ts-0/wal/instance:
uuid: "9014e5969a5342d987cedce48d96c2b5"
format_stamp: "Formatted at 2026-08-09 16:20:33 on dist-test-slave-s11t"
server_key: "ba8ef8e230e3f285343f1a6dd4913961"
server_key_iv: "5b9fbab93d7c011607763ea5567d6313"
server_key_version: "encryptionkey@0"
I20260809 16:20:33.010898 25589 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.001s
I20260809 16:20:33.011708 25613 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:20:33.011911 25589 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:20:33.012001 25589 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786292417722232-21710-0/minicluster-data/ts-0/data,/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786292417722232-21710-0/minicluster-data/ts-0/wal
uuid: "9014e5969a5342d987cedce48d96c2b5"
format_stamp: "Formatted at 2026-08-09 16:20:33 on dist-test-slave-s11t"
server_key: "ba8ef8e230e3f285343f1a6dd4913961"
server_key_iv: "5b9fbab93d7c011607763ea5567d6313"
server_key_version: "encryptionkey@0"
I20260809 16:20:33.012056 25589 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786292417722232-21710-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786292417722232-21710-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786292417722232-21710-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:20:33.047138 25589 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:20:33.049377 25589 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:20:33.049500 25589 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:20:33.049696 25589 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:20:33.050045 25589 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:20:33.050083 25589 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:20:33.050107 25589 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:20:33.050119 25589 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:20:33.056137 25589 rpc_server.cc:307] RPC server started. Bound to: 127.21.51.129:34885
I20260809 16:20:33.056181 25795 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.51.129:34885 every 8 connection(s)
I20260809 16:20:33.056502 25589 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786292417722232-21710-0/minicluster-data/ts-0/data/info.pb
I20260809 16:20:33.057432 21710 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrNqUCn/build/release/bin/kudu as pid 25589
I20260809 16:20:33.057535 21710 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786292417722232-21710-0/minicluster-data/ts-0/wal/instance
I20260809 16:20:33.057751 21710 external_mini_cluster.cc:1428] Setting key 90a4d2c81ac9d8af1e153047febb134b
Aug 09 16:20:33 dist-test-slave-s11t krb5kdc[25424](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292433, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.51.129@KRBTEST.COM for kudu/127.21.51.190@KRBTEST.COM
I20260809 16:20:33.067343 25571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:33.057335 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.51.129:43695 (local address 127.21.51.190:33739)
0809 16:20:33.057419 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0809 16:20:33.057422 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 16:20:33.057649 (+ 227us) server_negotiation.cc:407] Connection header received
0809 16:20:33.058337 (+ 688us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:20:33.058338 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:20:33.058352 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:20:33.058416 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:20:33.059774 (+ 1358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:33.060338 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:33.061494 (+ 1156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:33.061715 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:33.061794 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:33.064372 (+ 2578us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:20:33.064373 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:20:33.064373 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:20:33.064381 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:20:33.065835 (+ 1454us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:20:33.066344 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:20:33.066345 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:20:33.066345 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:20:33.066401 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:20:33.066689 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:20:33.066690 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:20:33.066690 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:20:33.066792 (+ 102us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:20:33.066843 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0809 16:20:33.067197 (+ 354us) server_negotiation.cc:299] Negotiation successful
0809 16:20:33.067276 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260809 16:20:33.067617 25799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:33.057237 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:33739 (local address 127.21.51.129:43695)
0809 16:20:33.057509 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0809 16:20:33.057563 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0809 16:20:33.058234 (+ 671us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:20:33.058503 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:20:33.058507 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:20:33.059386 (+ 879us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:20:33.059666 (+ 280us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:20:33.059672 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:20:33.060442 (+ 770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:20:33.060443 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:20:33.061398 (+ 955us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:20:33.061401 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:20:33.061805 (+ 404us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:20:33.061807 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:20:33.061932 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:33.062505 (+ 573us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:20:33.062513 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:20:33.064277 (+ 1764us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:20:33.065931 (+ 1654us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:20:33.065938 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:20:33.065941 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:20:33.066265 (+ 324us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:20:33.066474 (+ 209us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:20:33.066474 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:20:33.066475 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:20:33.066620 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:20:33.066863 (+ 243us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:20:33.066869 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:20:33.067043 (+ 174us) client_negotiation.cc:769] Sending connection context
0809 16:20:33.067179 (+ 136us) client_negotiation.cc:240] Negotiation successful
0809 16:20:33.067370 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":225,"thread_start_us":109,"threads_started":1}
WARNING: no policy specified for kudu/127.21.51.130@KRBTEST.COM; defaulting to no policy
I20260809 16:20:33.068302 25796 heartbeater.cc:347] Connected to a master server at 127.21.51.190:33739
I20260809 16:20:33.068409 25796 heartbeater.cc:464] Registering TS with master...
I20260809 16:20:33.068609 25796 heartbeater.cc:511] Master 127.21.51.190:33739 requested a full tablet report, sending...
I20260809 16:20:33.069274 25491 ts_manager.cc:194] Registered new tserver with Master: 9014e5969a5342d987cedce48d96c2b5 (127.21.51.129:34885)
I20260809 16:20:33.069859 25491 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.51.129@KRBTEST.COM'} at 127.21.51.129:43695
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.51.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.51.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.130.keytab.
Entry for principal kudu/127.21.51.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.130.keytab.
I20260809 16:20:33.085789 21710 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrNqUCn/build/release/bin/kudu
/tmp/dist-test-taskrNqUCn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786292417722232-21710-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786292417722232-21710-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786292417722232-21710-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786292417722232-21710-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.51.130:0
--local_ip_for_outbound_sockets=127.21.51.130
--webserver_interface=127.21.51.130
--webserver_port=0
--tserver_master_addrs=127.21.51.190:33739
--keytab_file=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.130.keytab
--principal=kudu/127.21.51.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.51.148:44071
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786292417722232-21710-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786292417722232-21710-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786292417722232-21710-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:20:33.168960 25804 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:20:33.169152 25804 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:20:33.169174 25804 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:20:33.170365 25804 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:20:33.170408 25804 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:20:33.170433 25804 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.51.130
I20260809 16:20:33.171844 25804 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.51.148:44071
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786292417722232-21710-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786292417722232-21710-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.130.keytab
--principal=kudu/127.21.51.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.51.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786292417722232-21710-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.51.130
--webserver_port=0
--tserver_master_addrs=127.21.51.190:33739
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25804
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.51.130
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786292417722232-21710-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:20:33.172089 25804 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:20:33.172330 25804 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:20:33.177388 25812 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:20:33.177397 25816 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:20:33.177402 25811 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:20:33.177819 25804 server_base.cc:1034] running on GCE node
I20260809 16:20:33.178047 25804 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:20:33.178284 25804 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:20:33.179442 25804 hybrid_clock.cc:648] HybridClock initialized: now 1786292433179434 us; error 33 us; skew 500 ppm
Aug 09 16:20:33 dist-test-slave-s11t krb5kdc[25424](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292433, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.51.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:20:33.182085 25804 init.cc:377] Logged in from keytab as kudu/127.21.51.130@KRBTEST.COM (short username kudu)
I20260809 16:20:33.182523 25804 webserver.cc:460] Webserver started at http://127.21.51.130:40221/ using document root <none> and password file <none>
I20260809 16:20:33.182715 25804 fs_manager.cc:359] Metadata directory not provided
I20260809 16:20:33.182755 25804 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:20:33.182845 25804 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:20:33.183816 25804 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786292417722232-21710-0/minicluster-data/ts-1/data/instance:
uuid: "5773fc455909415b96f5b1f8299d1f88"
format_stamp: "Formatted at 2026-08-09 16:20:33 on dist-test-slave-s11t"
server_key: "dfa297b890892161d0ad9d8981b3a48f"
server_key_iv: "3c73719aa30cc30654e377d66658a52c"
server_key_version: "encryptionkey@0"
I20260809 16:20:33.184126 25804 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786292417722232-21710-0/minicluster-data/ts-1/wal/instance:
uuid: "5773fc455909415b96f5b1f8299d1f88"
format_stamp: "Formatted at 2026-08-09 16:20:33 on dist-test-slave-s11t"
server_key: "dfa297b890892161d0ad9d8981b3a48f"
server_key_iv: "3c73719aa30cc30654e377d66658a52c"
server_key_version: "encryptionkey@0"
I20260809 16:20:33.185482 25804 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 16:20:33.186177 25828 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:20:33.186365 25804 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 16:20:33.186437 25804 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786292417722232-21710-0/minicluster-data/ts-1/data,/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786292417722232-21710-0/minicluster-data/ts-1/wal
uuid: "5773fc455909415b96f5b1f8299d1f88"
format_stamp: "Formatted at 2026-08-09 16:20:33 on dist-test-slave-s11t"
server_key: "dfa297b890892161d0ad9d8981b3a48f"
server_key_iv: "3c73719aa30cc30654e377d66658a52c"
server_key_version: "encryptionkey@0"
I20260809 16:20:33.186488 25804 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786292417722232-21710-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786292417722232-21710-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786292417722232-21710-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:20:33.203456 25804 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:20:33.206171 25804 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:20:33.206313 25804 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:20:33.206568 25804 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:20:33.206871 25804 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:20:33.206912 25804 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:20:33.206964 25804 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:20:33.206990 25804 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:20:33.213096 25804 rpc_server.cc:307] RPC server started. Bound to: 127.21.51.130:42057
I20260809 16:20:33.213148 26011 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.51.130:42057 every 8 connection(s)
I20260809 16:20:33.213455 25804 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786292417722232-21710-0/minicluster-data/ts-1/data/info.pb
Aug 09 16:20:33 dist-test-slave-s11t krb5kdc[25424](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292433, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.51.130@KRBTEST.COM for kudu/127.21.51.190@KRBTEST.COM
I20260809 16:20:33.220875 21710 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrNqUCn/build/release/bin/kudu as pid 25804
I20260809 16:20:33.220978 21710 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786292417722232-21710-0/minicluster-data/ts-1/wal/instance
I20260809 16:20:33.221172 21710 external_mini_cluster.cc:1428] Setting key f588bd92baa30b4bfa87b7a3ab998ea5
I20260809 16:20:33.223295 25571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:33.214140 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.51.130:55007 (local address 127.21.51.190:33739)
0809 16:20:33.214234 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0809 16:20:33.214237 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 16:20:33.214395 (+ 158us) server_negotiation.cc:407] Connection header received
0809 16:20:33.214967 (+ 572us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:20:33.214969 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:20:33.214980 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:20:33.215038 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:20:33.215968 (+ 930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:33.216529 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:33.217526 (+ 997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:33.217754 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:33.217816 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:33.219823 (+ 2007us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:20:33.219824 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:20:33.219825 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:20:33.219832 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:20:33.221695 (+ 1863us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:20:33.222247 (+ 552us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:20:33.222248 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:20:33.222248 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:20:33.222309 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:20:33.222605 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:20:33.222606 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:20:33.222606 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:20:33.222726 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:20:33.222789 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0809 16:20:33.223125 (+ 336us) server_negotiation.cc:299] Negotiation successful
0809 16:20:33.223225 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260809 16:20:33.223538 26015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:33.214089 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:33739 (local address 127.21.51.130:55007)
0809 16:20:33.214277 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0809 16:20:33.214312 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0809 16:20:33.214841 (+ 529us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:20:33.215086 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:20:33.215089 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:20:33.215629 (+ 540us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:20:33.215882 (+ 253us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:20:33.215887 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:20:33.216613 (+ 726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:20:33.216614 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:20:33.217452 (+ 838us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:20:33.217454 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:20:33.217832 (+ 378us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:20:33.217833 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:20:33.217931 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:33.218365 (+ 434us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:20:33.218381 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:20:33.219739 (+ 1358us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:20:33.221802 (+ 2063us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:20:33.221810 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:20:33.221812 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:20:33.222148 (+ 336us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:20:33.222390 (+ 242us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:20:33.222391 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:20:33.222391 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:20:33.222535 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:20:33.222800 (+ 265us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:20:33.222806 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:20:33.222962 (+ 156us) client_negotiation.cc:769] Sending connection context
0809 16:20:33.223080 (+ 118us) client_negotiation.cc:240] Negotiation successful
0809 16:20:33.223250 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":108,"threads_started":1}
I20260809 16:20:33.224045 26012 heartbeater.cc:347] Connected to a master server at 127.21.51.190:33739
I20260809 16:20:33.224153 26012 heartbeater.cc:464] Registering TS with master...
I20260809 16:20:33.224323 26012 heartbeater.cc:511] Master 127.21.51.190:33739 requested a full tablet report, sending...
I20260809 16:20:33.224773 25491 ts_manager.cc:194] Registered new tserver with Master: 5773fc455909415b96f5b1f8299d1f88 (127.21.51.130:42057)
I20260809 16:20:33.225318 25491 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.51.130@KRBTEST.COM'} at 127.21.51.130:55007
WARNING: no policy specified for kudu/127.21.51.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.51.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.51.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.131.keytab.
Entry for principal kudu/127.21.51.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.131.keytab.
I20260809 16:20:33.249046 21710 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrNqUCn/build/release/bin/kudu
/tmp/dist-test-taskrNqUCn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786292417722232-21710-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786292417722232-21710-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786292417722232-21710-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786292417722232-21710-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.51.131:0
--local_ip_for_outbound_sockets=127.21.51.131
--webserver_interface=127.21.51.131
--webserver_port=0
--tserver_master_addrs=127.21.51.190:33739
--keytab_file=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.131.keytab
--principal=kudu/127.21.51.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.51.148:44071
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786292417722232-21710-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786292417722232-21710-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786292417722232-21710-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:20:33.330426 26021 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:20:33.330653 26021 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:20:33.330689 26021 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:20:33.331821 26021 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:20:33.331885 26021 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:20:33.331929 26021 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.51.131
I20260809 16:20:33.333384 26021 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.51.148:44071
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786292417722232-21710-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786292417722232-21710-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.131.keytab
--principal=kudu/127.21.51.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.51.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786292417722232-21710-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.51.131
--webserver_port=0
--tserver_master_addrs=127.21.51.190:33739
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26021
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.51.131
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786292417722232-21710-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:20:33.333643 26021 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:20:33.333875 26021 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:20:33.338940 26028 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:20:33.338966 26029 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:20:33.338951 26031 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:20:33.339118 26021 server_base.cc:1034] running on GCE node
I20260809 16:20:33.339375 26021 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:20:33.339684 26021 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:20:33.340829 26021 hybrid_clock.cc:648] HybridClock initialized: now 1786292433340815 us; error 30 us; skew 500 ppm
Aug 09 16:20:33 dist-test-slave-s11t krb5kdc[25424](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292433, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.51.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:20:33.343467 26021 init.cc:377] Logged in from keytab as kudu/127.21.51.131@KRBTEST.COM (short username kudu)
I20260809 16:20:33.343847 26021 webserver.cc:460] Webserver started at http://127.21.51.131:43957/ using document root <none> and password file <none>
I20260809 16:20:33.344038 26021 fs_manager.cc:359] Metadata directory not provided
I20260809 16:20:33.344079 26021 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:20:33.344165 26021 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:20:33.345208 26021 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786292417722232-21710-0/minicluster-data/ts-2/data/instance:
uuid: "d1debedb78b94bee8b9da761d27c4f94"
format_stamp: "Formatted at 2026-08-09 16:20:33 on dist-test-slave-s11t"
server_key: "c8dfb495d2709b14313de6a232a26ac3"
server_key_iv: "051c063879088565d300710bc0d6c2de"
server_key_version: "encryptionkey@0"
I20260809 16:20:33.345512 26021 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786292417722232-21710-0/minicluster-data/ts-2/wal/instance:
uuid: "d1debedb78b94bee8b9da761d27c4f94"
format_stamp: "Formatted at 2026-08-09 16:20:33 on dist-test-slave-s11t"
server_key: "c8dfb495d2709b14313de6a232a26ac3"
server_key_iv: "051c063879088565d300710bc0d6c2de"
server_key_version: "encryptionkey@0"
I20260809 16:20:33.346819 26021 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 16:20:33.347551 26044 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:20:33.347754 26021 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:20:33.347862 26021 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786292417722232-21710-0/minicluster-data/ts-2/data,/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786292417722232-21710-0/minicluster-data/ts-2/wal
uuid: "d1debedb78b94bee8b9da761d27c4f94"
format_stamp: "Formatted at 2026-08-09 16:20:33 on dist-test-slave-s11t"
server_key: "c8dfb495d2709b14313de6a232a26ac3"
server_key_iv: "051c063879088565d300710bc0d6c2de"
server_key_version: "encryptionkey@0"
I20260809 16:20:33.347919 26021 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786292417722232-21710-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786292417722232-21710-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786292417722232-21710-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:20:33.356784 26021 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:20:33.358477 26021 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:20:33.358583 26021 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:20:33.358742 26021 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:20:33.359045 26021 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:20:33.359078 26021 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:20:33.359100 26021 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:20:33.359148 26021 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:20:33.365309 26021 rpc_server.cc:307] RPC server started. Bound to: 127.21.51.131:34387
I20260809 16:20:33.365389 26217 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.51.131:34387 every 8 connection(s)
I20260809 16:20:33.365777 26021 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786292417722232-21710-0/minicluster-data/ts-2/data/info.pb
Aug 09 16:20:33 dist-test-slave-s11t krb5kdc[25424](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292433, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.51.131@KRBTEST.COM for kudu/127.21.51.190@KRBTEST.COM
I20260809 16:20:33.373881 21710 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrNqUCn/build/release/bin/kudu as pid 26021
I20260809 16:20:33.373981 21710 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786292417722232-21710-0/minicluster-data/ts-2/wal/instance
I20260809 16:20:33.374172 21710 external_mini_cluster.cc:1428] Setting key e2f59ebff85ab13e1b17cc88188840e9
I20260809 16:20:33.375437 25571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:33.366561 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.51.131:60833 (local address 127.21.51.190:33739)
0809 16:20:33.366605 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0809 16:20:33.366607 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:20:33.366826 (+ 219us) server_negotiation.cc:407] Connection header received
0809 16:20:33.367349 (+ 523us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:20:33.367350 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:20:33.367365 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:20:33.367422 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:20:33.368358 (+ 936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:33.368955 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:33.369961 (+ 1006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:33.370154 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:33.370228 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:33.372399 (+ 2171us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:20:33.372401 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:20:33.372401 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:20:33.372409 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:20:33.373731 (+ 1322us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:20:33.374346 (+ 615us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:20:33.374348 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:20:33.374348 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:20:33.374419 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:20:33.374724 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:20:33.374725 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:20:33.374725 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:20:33.374914 (+ 189us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:20:33.374983 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0809 16:20:33.375304 (+ 321us) server_negotiation.cc:299] Negotiation successful
0809 16:20:33.375375 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260809 16:20:33.375579 26222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:33.366533 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:33739 (local address 127.21.51.131:60833)
0809 16:20:33.366707 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0809 16:20:33.366741 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0809 16:20:33.367233 (+ 492us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:20:33.367495 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:20:33.367499 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:20:33.368071 (+ 572us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:20:33.368254 (+ 183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:20:33.368258 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:20:33.369048 (+ 790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:20:33.369049 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:20:33.369857 (+ 808us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:20:33.369859 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:20:33.370236 (+ 377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:20:33.370237 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:20:33.370375 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:33.370903 (+ 528us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:20:33.370911 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:20:33.372309 (+ 1398us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:20:33.373868 (+ 1559us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:20:33.373884 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:20:33.373886 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:20:33.374207 (+ 321us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:20:33.374494 (+ 287us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:20:33.374495 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:20:33.374495 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:20:33.374656 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:20:33.374984 (+ 328us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:20:33.374989 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:20:33.375162 (+ 173us) client_negotiation.cc:769] Sending connection context
0809 16:20:33.375265 (+ 103us) client_negotiation.cc:240] Negotiation successful
0809 16:20:33.375391 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":84,"threads_started":1}
I20260809 16:20:33.376040 26219 heartbeater.cc:347] Connected to a master server at 127.21.51.190:33739
I20260809 16:20:33.376144 26219 heartbeater.cc:464] Registering TS with master...
I20260809 16:20:33.376317 26219 heartbeater.cc:511] Master 127.21.51.190:33739 requested a full tablet report, sending...
I20260809 16:20:33.376789 25491 ts_manager.cc:194] Registered new tserver with Master: d1debedb78b94bee8b9da761d27c4f94 (127.21.51.131:34387)
I20260809 16:20:33.377125 25491 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.51.131@KRBTEST.COM'} at 127.21.51.131:60833
I20260809 16:20:33.386035 21710 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260809 16:20:33.398263 25571 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:20:33.397359 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41474 (local address 127.21.51.190:33739)
0809 16:20:33.397411 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0809 16:20:33.397414 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 16:20:33.397461 (+ 47us) server_negotiation.cc:407] Connection header received
0809 16:20:33.397544 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:20:33.397545 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:20:33.397561 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:20:33.397625 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:20:33.398174 (+ 549us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41474: BlockingRecv error: recv got EOF from 127.0.0.1:41474 (error 108)
Metrics: {"server-negotiator.queue_time_us":25}
I20260809 16:20:33.399154 21710 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrNqUCn/build/release/bin/kudu with pid 25589
I20260809 16:20:33.424790 21710 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrNqUCn/build/release/bin/kudu with pid 25804
I20260809 16:20:33.432432 21710 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrNqUCn/build/release/bin/kudu with pid 26021
I20260809 16:20:33.447433 21710 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrNqUCn/build/release/bin/kudu with pid 25445
2026-08-09T16:20:33Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (2079 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786292417722232-21710-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 16:20:33 dist-test-slave-s11t krb5kdc[26237](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 16:20:33 dist-test-slave-s11t krb5kdc[26237](info): set up 2 sockets
Aug 09 16:20:33 dist-test-slave-s11t krb5kdc[26237](info): commencing operation
krb5kdc: starting...
W20260809 16:20:34.736662 21710 mini_kdc.cc:121] Time spent starting KDC: real 1.257s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786292417722232-21710-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786292417722232-21710-0/krb5kdc/test-user.keytab.
Aug 09 16:20:34 dist-test-slave-s11t krb5kdc[26237](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292434, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T16:20:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T16:20:34Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.51.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.51.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.51.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.190.keytab.
Entry for principal kudu/127.21.51.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.190.keytab.
I20260809 16:20:34.854790 21710 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrNqUCn/build/release/bin/kudu
/tmp/dist-test-taskrNqUCn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786292417722232-21710-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786292417722232-21710-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786292417722232-21710-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786292417722232-21710-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.51.190:40823
--webserver_interface=127.21.51.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.190.keytab
--principal=kudu/127.21.51.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.51.148:36975
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.51.190:40823
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786292417722232-21710-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786292417722232-21710-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786292417722232-21710-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:20:34.935596 26261 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:20:34.935830 26261 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:20:34.935868 26261 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:20:34.937039 26261 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 16:20:34.937108 26261 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:20:34.937131 26261 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:20:34.937150 26261 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 16:20:34.937181 26261 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 16:20:34.938654 26261 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.51.148:36975
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786292417722232-21710-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786292417722232-21710-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.51.190:40823
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.190.keytab
--principal=kudu/127.21.51.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.51.190:40823
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786292417722232-21710-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.51.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26261
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786292417722232-21710-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:20:34.938920 26261 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:20:34.939144 26261 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:20:34.944195 26276 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:20:34.944211 26270 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:20:34.944305 26261 server_base.cc:1034] running on GCE node
W20260809 16:20:34.944195 26271 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:20:34.944751 26261 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:20:34.945065 26261 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:20:34.946234 26261 hybrid_clock.cc:648] HybridClock initialized: now 1786292434946211 us; error 36 us; skew 500 ppm
Aug 09 16:20:34 dist-test-slave-s11t krb5kdc[26237](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292434, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.51.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:20:34.953722 26261 init.cc:377] Logged in from keytab as kudu/127.21.51.190@KRBTEST.COM (short username kudu)
I20260809 16:20:34.954149 26261 webserver.cc:460] Webserver started at http://127.21.51.190:41015/ using document root <none> and password file <none>
I20260809 16:20:34.954382 26261 fs_manager.cc:359] Metadata directory not provided
I20260809 16:20:34.954452 26261 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:20:34.954607 26261 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:20:34.955638 26261 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786292417722232-21710-0/minicluster-data/master-0/data/instance:
uuid: "9b8672f473f3415999afc39d3efb6a02"
format_stamp: "Formatted at 2026-08-09 16:20:34 on dist-test-slave-s11t"
server_key: "f36b178afafcc9055d083482b588e7b7"
server_key_iv: "5c3943b6351b679ee581c13aca89b9bf"
server_key_version: "encryptionkey@0"
I20260809 16:20:34.956012 26261 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786292417722232-21710-0/minicluster-data/master-0/wal/instance:
uuid: "9b8672f473f3415999afc39d3efb6a02"
format_stamp: "Formatted at 2026-08-09 16:20:34 on dist-test-slave-s11t"
server_key: "f36b178afafcc9055d083482b588e7b7"
server_key_iv: "5c3943b6351b679ee581c13aca89b9bf"
server_key_version: "encryptionkey@0"
I20260809 16:20:34.957504 26261 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 16:20:34.958312 26287 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:20:34.958513 26261 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 16:20:34.958629 26261 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786292417722232-21710-0/minicluster-data/master-0/data,/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786292417722232-21710-0/minicluster-data/master-0/wal
uuid: "9b8672f473f3415999afc39d3efb6a02"
format_stamp: "Formatted at 2026-08-09 16:20:34 on dist-test-slave-s11t"
server_key: "f36b178afafcc9055d083482b588e7b7"
server_key_iv: "5c3943b6351b679ee581c13aca89b9bf"
server_key_version: "encryptionkey@0"
I20260809 16:20:34.958709 26261 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786292417722232-21710-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786292417722232-21710-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786292417722232-21710-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:20:34.978698 26261 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:20:34.981153 26261 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:20:34.981338 26261 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:20:34.985440 26261 rpc_server.cc:307] RPC server started. Bound to: 127.21.51.190:40823
I20260809 16:20:34.985504 26364 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.51.190:40823 every 8 connection(s)
I20260809 16:20:34.986064 26261 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786292417722232-21710-0/minicluster-data/master-0/data/info.pb
I20260809 16:20:34.986819 26367 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:20:34.989151 26367 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9b8672f473f3415999afc39d3efb6a02: Bootstrap starting.
I20260809 16:20:34.989662 21710 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrNqUCn/build/release/bin/kudu as pid 26261
I20260809 16:20:34.989791 21710 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786292417722232-21710-0/minicluster-data/master-0/wal/instance
I20260809 16:20:34.989903 26367 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9b8672f473f3415999afc39d3efb6a02: Neither blocks nor log segments found. Creating new log.
I20260809 16:20:34.990150 21710 external_mini_cluster.cc:1428] Setting key d9413da0d0d6e32f77221ea89fa2cd9d
I20260809 16:20:34.990301 26367 log.cc:824] T 00000000000000000000000000000000 P 9b8672f473f3415999afc39d3efb6a02: Log is configured to *not* fsync() on all Append() calls
I20260809 16:20:34.991120 26367 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9b8672f473f3415999afc39d3efb6a02: No bootstrap required, opened a new log
I20260809 16:20:34.993155 26367 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9b8672f473f3415999afc39d3efb6a02 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b8672f473f3415999afc39d3efb6a02" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 40823 } }
I20260809 16:20:34.993311 26367 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9b8672f473f3415999afc39d3efb6a02 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:20:34.993333 26367 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9b8672f473f3415999afc39d3efb6a02 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9b8672f473f3415999afc39d3efb6a02, State: Initialized, Role: FOLLOWER
I20260809 16:20:34.993492 26367 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9b8672f473f3415999afc39d3efb6a02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b8672f473f3415999afc39d3efb6a02" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 40823 } }
I20260809 16:20:34.993568 26367 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9b8672f473f3415999afc39d3efb6a02 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 16:20:34.993590 26367 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9b8672f473f3415999afc39d3efb6a02 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 16:20:34.993655 26367 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9b8672f473f3415999afc39d3efb6a02 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:20:34.994201 26367 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9b8672f473f3415999afc39d3efb6a02 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b8672f473f3415999afc39d3efb6a02" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 40823 } }
I20260809 16:20:34.994292 26367 leader_election.cc:302] T 00000000000000000000000000000000 P 9b8672f473f3415999afc39d3efb6a02 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9b8672f473f3415999afc39d3efb6a02; no voters:
I20260809 16:20:34.994495 26367 leader_election.cc:288] T 00000000000000000000000000000000 P 9b8672f473f3415999afc39d3efb6a02 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 16:20:34.994522 26374 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9b8672f473f3415999afc39d3efb6a02 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:20:34.994635 26374 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9b8672f473f3415999afc39d3efb6a02 [term 1 LEADER]: Becoming Leader. State: Replica: 9b8672f473f3415999afc39d3efb6a02, State: Running, Role: LEADER
I20260809 16:20:34.994777 26367 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9b8672f473f3415999afc39d3efb6a02 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 16:20:34.994791 26374 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9b8672f473f3415999afc39d3efb6a02 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b8672f473f3415999afc39d3efb6a02" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 40823 } }
I20260809 16:20:34.996786 26375 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9b8672f473f3415999afc39d3efb6a02 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9b8672f473f3415999afc39d3efb6a02. Latest consensus state: current_term: 1 leader_uuid: "9b8672f473f3415999afc39d3efb6a02" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b8672f473f3415999afc39d3efb6a02" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 40823 } } }
I20260809 16:20:34.996953 26375 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9b8672f473f3415999afc39d3efb6a02 [sys.catalog]: This master's current role is: LEADER
I20260809 16:20:34.996968 26378 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9b8672f473f3415999afc39d3efb6a02 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9b8672f473f3415999afc39d3efb6a02" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b8672f473f3415999afc39d3efb6a02" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 40823 } } }
I20260809 16:20:34.997041 26378 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9b8672f473f3415999afc39d3efb6a02 [sys.catalog]: This master's current role is: LEADER
Aug 09 16:20:34 dist-test-slave-s11t krb5kdc[26237](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292434, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.51.190@KRBTEST.COM
W20260809 16:20:34.997787 26395 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 9b8672f473f3415999afc39d3efb6a02: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 16:20:34.997889 26395 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 16:20:34.997969 26399 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 16:20:34.998164 26399 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 16:20:34.999614 26399 catalog_manager.cc:1347] Generated new cluster ID: d6447a7889974031a203ef1b709529a8
I20260809 16:20:34.999670 26399 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 16:20:35.001955 26371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:34.990689 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58240 (local address 127.21.51.190:40823)
0809 16:20:34.990875 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0809 16:20:34.990884 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 16:20:34.990923 (+ 39us) server_negotiation.cc:407] Connection header received
0809 16:20:34.991412 (+ 489us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:20:34.991419 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:20:34.991468 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:20:34.991618 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:20:34.992280 (+ 662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:34.993236 (+ 956us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:34.994177 (+ 941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:34.994763 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:34.994867 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:34.997890 (+ 3023us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:20:34.997893 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:20:34.997894 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:20:34.997904 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:20:34.999940 (+ 2036us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:20:35.000351 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:20:35.000355 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:20:35.000355 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:20:35.000453 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:20:35.000661 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:20:35.000662 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:20:35.000662 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:20:35.000995 (+ 333us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:20:35.001137 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0809 16:20:35.001579 (+ 442us) server_negotiation.cc:299] Negotiation successful
0809 16:20:35.001718 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":134,"thread_start_us":103,"threads_started":1}
I20260809 16:20:35.005708 21710 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 16:20:35.017262 26399 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 16:20:35.017939 26399 catalog_manager.cc:1504] Loading token signing keys...
I20260809 16:20:35.023046 26371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:35.018914 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58242 (local address 127.21.51.190:40823)
0809 16:20:35.018972 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0809 16:20:35.018974 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:20:35.019069 (+ 95us) server_negotiation.cc:407] Connection header received
0809 16:20:35.019134 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:20:35.019136 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:20:35.019146 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:20:35.019199 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:20:35.019617 (+ 418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:35.020361 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:35.021458 (+ 1097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:35.021764 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:35.021826 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:35.022537 (+ 711us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:20:35.022538 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:20:35.022538 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 16:20:35.022763 (+ 225us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:20:35.022780 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 16:20:35.022786 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:20:35.022840 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0809 16:20:35.022947 (+ 107us) server_negotiation.cc:299] Negotiation successful
0809 16:20:35.022972 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260809 16:20:35.024117 26399 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9b8672f473f3415999afc39d3efb6a02: Generated new TSK 0
I20260809 16:20:35.024288 26399 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 16:20:35.030936 21710 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrNqUCn/build/release/bin/kudu with pid 26261
2026-08-09T16:20:35Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (1590 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786292417722232-21710-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 16:20:35 dist-test-slave-s11t krb5kdc[26425](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 16:20:35 dist-test-slave-s11t krb5kdc[26425](info): set up 2 sockets
Aug 09 16:20:35 dist-test-slave-s11t krb5kdc[26425](info): commencing operation
krb5kdc: starting...
W20260809 16:20:36.326431 21710 mini_kdc.cc:121] Time spent starting KDC: real 1.257s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786292417722232-21710-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786292417722232-21710-0/krb5kdc/test-user.keytab.
Aug 09 16:20:36 dist-test-slave-s11t krb5kdc[26425](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292436, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T16:20:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T16:20:36Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.51.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.51.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.51.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.190.keytab.
Entry for principal kudu/127.21.51.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.190.keytab.
I20260809 16:20:36.455351 21710 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrNqUCn/build/release/bin/kudu
/tmp/dist-test-taskrNqUCn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786292417722232-21710-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786292417722232-21710-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786292417722232-21710-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786292417722232-21710-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.51.190:33143
--webserver_interface=127.21.51.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.190.keytab
--principal=kudu/127.21.51.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.51.148:42801
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.51.190:33143
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786292417722232-21710-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786292417722232-21710-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786292417722232-21710-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:20:36.538883 26443 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:20:36.539098 26443 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:20:36.539120 26443 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:20:36.540201 26443 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 16:20:36.540237 26443 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:20:36.540251 26443 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:20:36.540264 26443 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 16:20:36.540275 26443 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 16:20:36.541764 26443 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.51.148:42801
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786292417722232-21710-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786292417722232-21710-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.51.190:33143
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.190.keytab
--principal=kudu/127.21.51.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.51.190:33143
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786292417722232-21710-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.51.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26443
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786292417722232-21710-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:20:36.542035 26443 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:20:36.542276 26443 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:20:36.547333 26451 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:20:36.547340 26454 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:20:36.547365 26452 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:20:36.547710 26443 server_base.cc:1034] running on GCE node
I20260809 16:20:36.547945 26443 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:20:36.548192 26443 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:20:36.549330 26443 hybrid_clock.cc:648] HybridClock initialized: now 1786292436549311 us; error 43 us; skew 500 ppm
Aug 09 16:20:36 dist-test-slave-s11t krb5kdc[26425](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292436, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.51.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:20:36.551880 26443 init.cc:377] Logged in from keytab as kudu/127.21.51.190@KRBTEST.COM (short username kudu)
I20260809 16:20:36.552295 26443 webserver.cc:460] Webserver started at http://127.21.51.190:35373/ using document root <none> and password file <none>
I20260809 16:20:36.552489 26443 fs_manager.cc:359] Metadata directory not provided
I20260809 16:20:36.552534 26443 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:20:36.552630 26443 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:20:36.553784 26443 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786292417722232-21710-0/minicluster-data/master-0/data/instance:
uuid: "9d3801361d084ec5a54307f33f284b7b"
format_stamp: "Formatted at 2026-08-09 16:20:36 on dist-test-slave-s11t"
server_key: "53eaf2a85444500bf689b57b64081dcc"
server_key_iv: "9776fd54e183de7f6895988d42c9e182"
server_key_version: "encryptionkey@0"
I20260809 16:20:36.554113 26443 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786292417722232-21710-0/minicluster-data/master-0/wal/instance:
uuid: "9d3801361d084ec5a54307f33f284b7b"
format_stamp: "Formatted at 2026-08-09 16:20:36 on dist-test-slave-s11t"
server_key: "53eaf2a85444500bf689b57b64081dcc"
server_key_iv: "9776fd54e183de7f6895988d42c9e182"
server_key_version: "encryptionkey@0"
I20260809 16:20:36.555477 26443 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 16:20:36.556205 26468 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:20:36.556396 26443 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 16:20:36.556478 26443 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786292417722232-21710-0/minicluster-data/master-0/data,/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786292417722232-21710-0/minicluster-data/master-0/wal
uuid: "9d3801361d084ec5a54307f33f284b7b"
format_stamp: "Formatted at 2026-08-09 16:20:36 on dist-test-slave-s11t"
server_key: "53eaf2a85444500bf689b57b64081dcc"
server_key_iv: "9776fd54e183de7f6895988d42c9e182"
server_key_version: "encryptionkey@0"
I20260809 16:20:36.556533 26443 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786292417722232-21710-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786292417722232-21710-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786292417722232-21710-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:20:36.582324 26443 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:20:36.584790 26443 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:20:36.584911 26443 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:20:36.589131 26443 rpc_server.cc:307] RPC server started. Bound to: 127.21.51.190:33143
I20260809 16:20:36.589166 26553 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.51.190:33143 every 8 connection(s)
I20260809 16:20:36.589535 26443 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786292417722232-21710-0/minicluster-data/master-0/data/info.pb
I20260809 16:20:36.590706 26554 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:20:36.591183 21710 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrNqUCn/build/release/bin/kudu as pid 26443
I20260809 16:20:36.591295 21710 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786292417722232-21710-0/minicluster-data/master-0/wal/instance
I20260809 16:20:36.591495 21710 external_mini_cluster.cc:1428] Setting key 79c0d8827e6e7a21dca39f514e2237e6
I20260809 16:20:36.594017 26554 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9d3801361d084ec5a54307f33f284b7b: Bootstrap starting.
I20260809 16:20:36.594635 26554 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9d3801361d084ec5a54307f33f284b7b: Neither blocks nor log segments found. Creating new log.
I20260809 16:20:36.594995 26554 log.cc:824] T 00000000000000000000000000000000 P 9d3801361d084ec5a54307f33f284b7b: Log is configured to *not* fsync() on all Append() calls
I20260809 16:20:36.595811 26554 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9d3801361d084ec5a54307f33f284b7b: No bootstrap required, opened a new log
I20260809 16:20:36.598407 26554 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9d3801361d084ec5a54307f33f284b7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d3801361d084ec5a54307f33f284b7b" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 33143 } }
I20260809 16:20:36.598544 26554 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9d3801361d084ec5a54307f33f284b7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:20:36.598567 26554 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9d3801361d084ec5a54307f33f284b7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9d3801361d084ec5a54307f33f284b7b, State: Initialized, Role: FOLLOWER
Aug 09 16:20:36 dist-test-slave-s11t krb5kdc[26425](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292436, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.51.190@KRBTEST.COM
I20260809 16:20:36.598713 26554 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9d3801361d084ec5a54307f33f284b7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d3801361d084ec5a54307f33f284b7b" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 33143 } }
I20260809 16:20:36.598793 26554 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9d3801361d084ec5a54307f33f284b7b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 16:20:36.598845 26554 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9d3801361d084ec5a54307f33f284b7b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 16:20:36.598906 26554 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9d3801361d084ec5a54307f33f284b7b [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:20:36.599506 26554 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9d3801361d084ec5a54307f33f284b7b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d3801361d084ec5a54307f33f284b7b" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 33143 } }
I20260809 16:20:36.599646 26554 leader_election.cc:302] T 00000000000000000000000000000000 P 9d3801361d084ec5a54307f33f284b7b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9d3801361d084ec5a54307f33f284b7b; no voters:
I20260809 16:20:36.599829 26554 leader_election.cc:288] T 00000000000000000000000000000000 P 9d3801361d084ec5a54307f33f284b7b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 16:20:36.600010 26560 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9d3801361d084ec5a54307f33f284b7b [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:20:36.600126 26554 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9d3801361d084ec5a54307f33f284b7b [sys.catalog]: configured and running, proceeding with master startup.
I20260809 16:20:36.600191 26560 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9d3801361d084ec5a54307f33f284b7b [term 1 LEADER]: Becoming Leader. State: Replica: 9d3801361d084ec5a54307f33f284b7b, State: Running, Role: LEADER
I20260809 16:20:36.600310 26560 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9d3801361d084ec5a54307f33f284b7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d3801361d084ec5a54307f33f284b7b" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 33143 } }
I20260809 16:20:36.600801 26563 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9d3801361d084ec5a54307f33f284b7b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9d3801361d084ec5a54307f33f284b7b. Latest consensus state: current_term: 1 leader_uuid: "9d3801361d084ec5a54307f33f284b7b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d3801361d084ec5a54307f33f284b7b" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 33143 } } }
I20260809 16:20:36.600836 26562 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9d3801361d084ec5a54307f33f284b7b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9d3801361d084ec5a54307f33f284b7b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d3801361d084ec5a54307f33f284b7b" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 33143 } } }
I20260809 16:20:36.600903 26563 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9d3801361d084ec5a54307f33f284b7b [sys.catalog]: This master's current role is: LEADER
I20260809 16:20:36.600907 26562 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9d3801361d084ec5a54307f33f284b7b [sys.catalog]: This master's current role is: LEADER
W20260809 16:20:36.602237 26581 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 9d3801361d084ec5a54307f33f284b7b: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 16:20:36.602296 26581 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 16:20:36.602448 26573 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 16:20:36.602630 26573 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 16:20:36.603194 26558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:36.592068 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44848 (local address 127.21.51.190:33143)
0809 16:20:36.592268 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0809 16:20:36.592276 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 16:20:36.592302 (+ 26us) server_negotiation.cc:407] Connection header received
0809 16:20:36.592821 (+ 519us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:20:36.592827 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:20:36.592879 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:20:36.592999 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:20:36.593808 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:36.594779 (+ 971us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:36.595819 (+ 1040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:36.596376 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:36.596464 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:36.599303 (+ 2839us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:20:36.599309 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:20:36.599309 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:20:36.599323 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:20:36.601428 (+ 2105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:20:36.601861 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:20:36.601864 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:20:36.601865 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:20:36.601968 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:20:36.602192 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:20:36.602193 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:20:36.602194 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:20:36.602520 (+ 326us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:20:36.602653 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0809 16:20:36.602836 (+ 183us) server_negotiation.cc:299] Negotiation successful
0809 16:20:36.602992 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":138,"thread_start_us":89,"threads_started":1}
I20260809 16:20:36.604494 26573 catalog_manager.cc:1347] Generated new cluster ID: 14cee11bddfd4a67ace281553ae81e83
I20260809 16:20:36.604552 26573 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 16:20:36.618583 26573 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 16:20:36.619311 26573 catalog_manager.cc:1504] Loading token signing keys...
I20260809 16:20:36.622651 26573 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9d3801361d084ec5a54307f33f284b7b: Generated new TSK 0
I20260809 16:20:36.622843 26573 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.21.51.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.51.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.51.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.129.keytab.
Entry for principal kudu/127.21.51.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.129.keytab.
I20260809 16:20:36.649283 21710 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrNqUCn/build/release/bin/kudu
/tmp/dist-test-taskrNqUCn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786292417722232-21710-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786292417722232-21710-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786292417722232-21710-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786292417722232-21710-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.51.129:0
--local_ip_for_outbound_sockets=127.21.51.129
--webserver_interface=127.21.51.129
--webserver_port=0
--tserver_master_addrs=127.21.51.190:33143
--keytab_file=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.129.keytab
--principal=kudu/127.21.51.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.51.148:42801
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786292417722232-21710-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786292417722232-21710-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786292417722232-21710-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:20:36.730165 26590 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:20:36.730396 26590 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:20:36.730432 26590 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:20:36.731527 26590 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:20:36.731593 26590 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:20:36.731638 26590 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.51.129
I20260809 16:20:36.733127 26590 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.51.148:42801
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786292417722232-21710-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786292417722232-21710-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.129.keytab
--principal=kudu/127.21.51.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.51.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786292417722232-21710-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.51.129
--webserver_port=0
--tserver_master_addrs=127.21.51.190:33143
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26590
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.51.129
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786292417722232-21710-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:20:36.733393 26590 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:20:36.733635 26590 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:20:36.738778 26597 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:20:36.738798 26602 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:20:36.738778 26600 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:20:36.739056 26590 server_base.cc:1034] running on GCE node
I20260809 16:20:36.739248 26590 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:20:36.739503 26590 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:20:36.740675 26590 hybrid_clock.cc:648] HybridClock initialized: now 1786292436740655 us; error 30 us; skew 500 ppm
Aug 09 16:20:36 dist-test-slave-s11t krb5kdc[26425](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292436, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.51.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:20:36.743259 26590 init.cc:377] Logged in from keytab as kudu/127.21.51.129@KRBTEST.COM (short username kudu)
I20260809 16:20:36.743818 26590 webserver.cc:460] Webserver started at http://127.21.51.129:44221/ using document root <none> and password file <none>
I20260809 16:20:36.744071 26590 fs_manager.cc:359] Metadata directory not provided
I20260809 16:20:36.744143 26590 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:20:36.744297 26590 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:20:36.745410 26590 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786292417722232-21710-0/minicluster-data/ts-0/data/instance:
uuid: "5555ce7ff54a4e9c906e3386a48ed9a2"
format_stamp: "Formatted at 2026-08-09 16:20:36 on dist-test-slave-s11t"
server_key: "12020e7b13ea95960d755d3f840c8279"
server_key_iv: "97904cef990e389bcaa1da3058d990eb"
server_key_version: "encryptionkey@0"
I20260809 16:20:36.745782 26590 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786292417722232-21710-0/minicluster-data/ts-0/wal/instance:
uuid: "5555ce7ff54a4e9c906e3386a48ed9a2"
format_stamp: "Formatted at 2026-08-09 16:20:36 on dist-test-slave-s11t"
server_key: "12020e7b13ea95960d755d3f840c8279"
server_key_iv: "97904cef990e389bcaa1da3058d990eb"
server_key_version: "encryptionkey@0"
I20260809 16:20:36.747143 26590 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 16:20:36.747846 26614 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:20:36.748016 26590 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 16:20:36.748122 26590 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786292417722232-21710-0/minicluster-data/ts-0/data,/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786292417722232-21710-0/minicluster-data/ts-0/wal
uuid: "5555ce7ff54a4e9c906e3386a48ed9a2"
format_stamp: "Formatted at 2026-08-09 16:20:36 on dist-test-slave-s11t"
server_key: "12020e7b13ea95960d755d3f840c8279"
server_key_iv: "97904cef990e389bcaa1da3058d990eb"
server_key_version: "encryptionkey@0"
I20260809 16:20:36.748198 26590 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786292417722232-21710-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786292417722232-21710-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786292417722232-21710-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:20:36.783658 26590 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:20:36.786119 26590 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:20:36.786293 26590 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:20:36.786540 26590 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:20:36.786877 26590 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:20:36.786934 26590 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:20:36.786980 26590 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:20:36.787014 26590 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:20:36.792876 26590 rpc_server.cc:307] RPC server started. Bound to: 127.21.51.129:41825
I20260809 16:20:36.792922 26782 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.51.129:41825 every 8 connection(s)
I20260809 16:20:36.793242 26590 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786292417722232-21710-0/minicluster-data/ts-0/data/info.pb
I20260809 16:20:36.795107 21710 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrNqUCn/build/release/bin/kudu as pid 26590
I20260809 16:20:36.795228 21710 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786292417722232-21710-0/minicluster-data/ts-0/wal/instance
I20260809 16:20:36.795421 21710 external_mini_cluster.cc:1428] Setting key 3828245139c0bfbc275f7715ae26a853
Aug 09 16:20:36 dist-test-slave-s11t krb5kdc[26425](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292436, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.51.129@KRBTEST.COM for kudu/127.21.51.190@KRBTEST.COM
I20260809 16:20:36.803718 26558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:36.793980 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.51.129:37715 (local address 127.21.51.190:33143)
0809 16:20:36.794031 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0809 16:20:36.794034 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 16:20:36.794208 (+ 174us) server_negotiation.cc:407] Connection header received
0809 16:20:36.794702 (+ 494us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:20:36.794703 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:20:36.794713 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:20:36.794776 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:20:36.795800 (+ 1024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:36.796531 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:36.797645 (+ 1114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:36.797987 (+ 342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:36.798085 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:36.800585 (+ 2500us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:20:36.800587 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:20:36.800587 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:20:36.800594 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:20:36.802165 (+ 1571us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:20:36.802712 (+ 547us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:20:36.802712 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:20:36.802713 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:20:36.802766 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:20:36.803080 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:20:36.803081 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:20:36.803081 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:20:36.803185 (+ 104us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:20:36.803237 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0809 16:20:36.803580 (+ 343us) server_negotiation.cc:299] Negotiation successful
0809 16:20:36.803654 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260809 16:20:36.804005 26786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:36.793921 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:33143 (local address 127.21.51.129:37715)
0809 16:20:36.794088 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0809 16:20:36.794131 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0809 16:20:36.794605 (+ 474us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:20:36.794827 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:20:36.794830 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:20:36.795465 (+ 635us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:20:36.795707 (+ 242us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:20:36.795718 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:20:36.796640 (+ 922us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:20:36.796641 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:20:36.797542 (+ 901us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:20:36.797545 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:20:36.798094 (+ 549us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:20:36.798095 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:20:36.798243 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:36.798793 (+ 550us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:20:36.798801 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:20:36.800500 (+ 1699us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:20:36.802255 (+ 1755us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:20:36.802262 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:20:36.802265 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:20:36.802618 (+ 353us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:20:36.802845 (+ 227us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:20:36.802846 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:20:36.802847 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:20:36.803005 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:20:36.803253 (+ 248us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:20:36.803260 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:20:36.803433 (+ 173us) client_negotiation.cc:769] Sending connection context
0809 16:20:36.803562 (+ 129us) client_negotiation.cc:240] Negotiation successful
0809 16:20:36.803740 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":98,"threads_started":1}
I20260809 16:20:36.804627 26783 heartbeater.cc:347] Connected to a master server at 127.21.51.190:33143
I20260809 16:20:36.804749 26783 heartbeater.cc:464] Registering TS with master...
I20260809 16:20:36.804976 26783 heartbeater.cc:511] Master 127.21.51.190:33143 requested a full tablet report, sending...
I20260809 16:20:36.805595 26498 ts_manager.cc:194] Registered new tserver with Master: 5555ce7ff54a4e9c906e3386a48ed9a2 (127.21.51.129:41825)
WARNING: no policy specified for kudu/127.21.51.130@KRBTEST.COM; defaulting to no policy
I20260809 16:20:36.806414 26498 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.51.129@KRBTEST.COM'} at 127.21.51.129:37715
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.51.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.51.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.130.keytab.
Entry for principal kudu/127.21.51.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.130.keytab.
I20260809 16:20:36.824015 21710 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrNqUCn/build/release/bin/kudu
/tmp/dist-test-taskrNqUCn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786292417722232-21710-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786292417722232-21710-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786292417722232-21710-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786292417722232-21710-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.51.130:0
--local_ip_for_outbound_sockets=127.21.51.130
--webserver_interface=127.21.51.130
--webserver_port=0
--tserver_master_addrs=127.21.51.190:33143
--keytab_file=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.130.keytab
--principal=kudu/127.21.51.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.51.148:42801
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786292417722232-21710-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786292417722232-21710-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786292417722232-21710-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:20:36.905671 26790 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:20:36.905861 26790 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:20:36.905885 26790 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:20:36.907027 26790 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:20:36.907071 26790 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:20:36.907096 26790 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.51.130
I20260809 16:20:36.908576 26790 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.51.148:42801
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786292417722232-21710-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786292417722232-21710-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.130.keytab
--principal=kudu/127.21.51.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.51.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786292417722232-21710-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.51.130
--webserver_port=0
--tserver_master_addrs=127.21.51.190:33143
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26790
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.51.130
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786292417722232-21710-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:20:36.908874 26790 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:20:36.909117 26790 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:20:36.914168 26798 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:20:36.914179 26797 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:20:36.914222 26802 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:20:36.914426 26790 server_base.cc:1034] running on GCE node
I20260809 16:20:36.914624 26790 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:20:36.914880 26790 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:20:36.916040 26790 hybrid_clock.cc:648] HybridClock initialized: now 1786292436916028 us; error 37 us; skew 500 ppm
Aug 09 16:20:36 dist-test-slave-s11t krb5kdc[26425](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292436, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.51.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:20:36.918756 26790 init.cc:377] Logged in from keytab as kudu/127.21.51.130@KRBTEST.COM (short username kudu)
I20260809 16:20:36.919139 26790 webserver.cc:460] Webserver started at http://127.21.51.130:39843/ using document root <none> and password file <none>
I20260809 16:20:36.919343 26790 fs_manager.cc:359] Metadata directory not provided
I20260809 16:20:36.919387 26790 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:20:36.919479 26790 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:20:36.920464 26790 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786292417722232-21710-0/minicluster-data/ts-1/data/instance:
uuid: "b0424d5922954366b9ea19f96d693b79"
format_stamp: "Formatted at 2026-08-09 16:20:36 on dist-test-slave-s11t"
server_key: "a189a3396bd8f069561a3f7a383606fc"
server_key_iv: "da7cd6941fa5a0419f2a7e7a5f4f4ba5"
server_key_version: "encryptionkey@0"
I20260809 16:20:36.920856 26790 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786292417722232-21710-0/minicluster-data/ts-1/wal/instance:
uuid: "b0424d5922954366b9ea19f96d693b79"
format_stamp: "Formatted at 2026-08-09 16:20:36 on dist-test-slave-s11t"
server_key: "a189a3396bd8f069561a3f7a383606fc"
server_key_iv: "da7cd6941fa5a0419f2a7e7a5f4f4ba5"
server_key_version: "encryptionkey@0"
I20260809 16:20:36.922174 26790 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:20:36.922904 26815 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:20:36.923074 26790 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 16:20:36.923143 26790 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786292417722232-21710-0/minicluster-data/ts-1/data,/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786292417722232-21710-0/minicluster-data/ts-1/wal
uuid: "b0424d5922954366b9ea19f96d693b79"
format_stamp: "Formatted at 2026-08-09 16:20:36 on dist-test-slave-s11t"
server_key: "a189a3396bd8f069561a3f7a383606fc"
server_key_iv: "da7cd6941fa5a0419f2a7e7a5f4f4ba5"
server_key_version: "encryptionkey@0"
I20260809 16:20:36.923233 26790 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786292417722232-21710-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786292417722232-21710-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786292417722232-21710-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:20:36.949661 26790 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:20:36.951678 26790 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:20:36.951839 26790 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:20:36.952078 26790 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:20:36.952414 26790 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:20:36.952448 26790 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:20:36.952471 26790 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:20:36.952518 26790 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:20:36.958629 26979 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.51.130:34145 every 8 connection(s)
I20260809 16:20:36.958627 26790 rpc_server.cc:307] RPC server started. Bound to: 127.21.51.130:34145
I20260809 16:20:36.959091 26790 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786292417722232-21710-0/minicluster-data/ts-1/data/info.pb
Aug 09 16:20:36 dist-test-slave-s11t krb5kdc[26425](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292436, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.51.130@KRBTEST.COM for kudu/127.21.51.190@KRBTEST.COM
I20260809 16:20:36.968261 26558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:36.959851 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.51.130:42691 (local address 127.21.51.190:33143)
0809 16:20:36.959909 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0809 16:20:36.959911 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:20:36.960122 (+ 211us) server_negotiation.cc:407] Connection header received
0809 16:20:36.960603 (+ 481us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:20:36.960604 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:20:36.960614 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:20:36.960673 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:20:36.961629 (+ 956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:36.962160 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:36.963191 (+ 1031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:36.963373 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:36.963426 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:36.965598 (+ 2172us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:20:36.965600 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:20:36.965600 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:20:36.965607 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:20:36.967020 (+ 1413us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:20:36.967427 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:20:36.967428 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:20:36.967429 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:20:36.967486 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:20:36.967723 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:20:36.967724 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:20:36.967724 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:20:36.967831 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:20:36.967892 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0809 16:20:36.968131 (+ 239us) server_negotiation.cc:299] Negotiation successful
0809 16:20:36.968197 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260809 16:20:36.968436 26983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:36.959823 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:33143 (local address 127.21.51.130:42691)
0809 16:20:36.959985 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0809 16:20:36.960026 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0809 16:20:36.960516 (+ 490us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:20:36.960783 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:20:36.960788 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:20:36.961357 (+ 569us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:20:36.961546 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:20:36.961551 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:20:36.962252 (+ 701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:20:36.962252 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:20:36.963108 (+ 856us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:20:36.963110 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:20:36.963450 (+ 340us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:20:36.963451 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:20:36.963568 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:36.964009 (+ 441us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:20:36.964018 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:20:36.965505 (+ 1487us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:20:36.967113 (+ 1608us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:20:36.967119 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:20:36.967121 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:20:36.967350 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:20:36.967560 (+ 210us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:20:36.967560 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:20:36.967560 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:20:36.967662 (+ 102us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:20:36.967894 (+ 232us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:20:36.967899 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:20:36.968013 (+ 114us) client_negotiation.cc:769] Sending connection context
0809 16:20:36.968099 (+ 86us) client_negotiation.cc:240] Negotiation successful
0809 16:20:36.968234 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":92,"threads_started":1}
I20260809 16:20:36.969192 21710 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrNqUCn/build/release/bin/kudu as pid 26790
I20260809 16:20:36.969208 26980 heartbeater.cc:347] Connected to a master server at 127.21.51.190:33143
I20260809 16:20:36.969316 21710 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786292417722232-21710-0/minicluster-data/ts-1/wal/instance
I20260809 16:20:36.969317 26980 heartbeater.cc:464] Registering TS with master...
I20260809 16:20:36.969506 26980 heartbeater.cc:511] Master 127.21.51.190:33143 requested a full tablet report, sending...
I20260809 16:20:36.969521 21710 external_mini_cluster.cc:1428] Setting key 8ba3891341f2da437c301550121c2cd6
I20260809 16:20:36.969902 26497 ts_manager.cc:194] Registered new tserver with Master: b0424d5922954366b9ea19f96d693b79 (127.21.51.130:34145)
I20260809 16:20:36.970372 26497 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.51.130@KRBTEST.COM'} at 127.21.51.130:42691
WARNING: no policy specified for kudu/127.21.51.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.51.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.51.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.131.keytab.
Entry for principal kudu/127.21.51.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.131.keytab.
I20260809 16:20:36.995373 21710 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrNqUCn/build/release/bin/kudu
/tmp/dist-test-taskrNqUCn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786292417722232-21710-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786292417722232-21710-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786292417722232-21710-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786292417722232-21710-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.51.131:0
--local_ip_for_outbound_sockets=127.21.51.131
--webserver_interface=127.21.51.131
--webserver_port=0
--tserver_master_addrs=127.21.51.190:33143
--keytab_file=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.131.keytab
--principal=kudu/127.21.51.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.51.148:42801
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786292417722232-21710-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786292417722232-21710-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786292417722232-21710-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:20:37.078511 26989 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:20:37.078744 26989 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:20:37.078783 26989 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:20:37.079907 26989 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:20:37.079973 26989 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:20:37.080026 26989 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.51.131
I20260809 16:20:37.081507 26989 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.51.148:42801
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786292417722232-21710-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786292417722232-21710-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.131.keytab
--principal=kudu/127.21.51.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.51.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786292417722232-21710-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.51.131
--webserver_port=0
--tserver_master_addrs=127.21.51.190:33143
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26989
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.51.131
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786292417722232-21710-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:20:37.081764 26989 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:20:37.082006 26989 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:20:37.087031 26997 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:20:37.087097 26996 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:20:37.087148 26989 server_base.cc:1034] running on GCE node
W20260809 16:20:37.087116 27001 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:20:37.087496 26989 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:20:37.087711 26989 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:20:37.088853 26989 hybrid_clock.cc:648] HybridClock initialized: now 1786292437088838 us; error 29 us; skew 500 ppm
Aug 09 16:20:37 dist-test-slave-s11t krb5kdc[26425](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292437, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.51.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:20:37.091403 26989 init.cc:377] Logged in from keytab as kudu/127.21.51.131@KRBTEST.COM (short username kudu)
I20260809 16:20:37.091835 26989 webserver.cc:460] Webserver started at http://127.21.51.131:33529/ using document root <none> and password file <none>
I20260809 16:20:37.092077 26989 fs_manager.cc:359] Metadata directory not provided
I20260809 16:20:37.092121 26989 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:20:37.092275 26989 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:20:37.093381 26989 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786292417722232-21710-0/minicluster-data/ts-2/data/instance:
uuid: "a307d365d363472fa4138911611010ed"
format_stamp: "Formatted at 2026-08-09 16:20:37 on dist-test-slave-s11t"
server_key: "6cc09006ec5c2d385c2f6f3257ac4ac6"
server_key_iv: "bfcf176dd85f23c3265808aa1785239e"
server_key_version: "encryptionkey@0"
I20260809 16:20:37.093748 26989 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786292417722232-21710-0/minicluster-data/ts-2/wal/instance:
uuid: "a307d365d363472fa4138911611010ed"
format_stamp: "Formatted at 2026-08-09 16:20:37 on dist-test-slave-s11t"
server_key: "6cc09006ec5c2d385c2f6f3257ac4ac6"
server_key_iv: "bfcf176dd85f23c3265808aa1785239e"
server_key_version: "encryptionkey@0"
I20260809 16:20:37.095099 26989 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 16:20:37.095903 27020 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:20:37.096110 26989 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:20:37.096186 26989 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786292417722232-21710-0/minicluster-data/ts-2/data,/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786292417722232-21710-0/minicluster-data/ts-2/wal
uuid: "a307d365d363472fa4138911611010ed"
format_stamp: "Formatted at 2026-08-09 16:20:37 on dist-test-slave-s11t"
server_key: "6cc09006ec5c2d385c2f6f3257ac4ac6"
server_key_iv: "bfcf176dd85f23c3265808aa1785239e"
server_key_version: "encryptionkey@0"
I20260809 16:20:37.096278 26989 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786292417722232-21710-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786292417722232-21710-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786292417722232-21710-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:20:37.110612 26989 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:20:37.113054 26989 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:20:37.113197 26989 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:20:37.113448 26989 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:20:37.113770 26989 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:20:37.113803 26989 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:20:37.113847 26989 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:20:37.113883 26989 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:20:37.120260 26989 rpc_server.cc:307] RPC server started. Bound to: 127.21.51.131:41841
I20260809 16:20:37.120321 27192 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.51.131:41841 every 8 connection(s)
I20260809 16:20:37.120666 26989 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786292417722232-21710-0/minicluster-data/ts-2/data/info.pb
Aug 09 16:20:37 dist-test-slave-s11t krb5kdc[26425](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292437, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.51.131@KRBTEST.COM for kudu/127.21.51.190@KRBTEST.COM
I20260809 16:20:37.130653 21710 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrNqUCn/build/release/bin/kudu as pid 26989
I20260809 16:20:37.130755 21710 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786292417722232-21710-0/minicluster-data/ts-2/wal/instance
I20260809 16:20:37.130859 26558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:37.121594 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.51.131:45127 (local address 127.21.51.190:33143)
0809 16:20:37.121667 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0809 16:20:37.121669 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:20:37.121857 (+ 188us) server_negotiation.cc:407] Connection header received
0809 16:20:37.122396 (+ 539us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:20:37.122397 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:20:37.122408 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:20:37.122470 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:20:37.123591 (+ 1121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:37.124110 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:37.125362 (+ 1252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:37.125556 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:37.125614 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:37.127948 (+ 2334us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:20:37.127949 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:20:37.127949 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:20:37.127956 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:20:37.129430 (+ 1474us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:20:37.129887 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:20:37.129888 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:20:37.129888 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:20:37.129948 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:20:37.130194 (+ 246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:20:37.130195 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:20:37.130195 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:20:37.130299 (+ 104us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:20:37.130355 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0809 16:20:37.130673 (+ 318us) server_negotiation.cc:299] Negotiation successful
0809 16:20:37.130763 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260809 16:20:37.130940 21710 external_mini_cluster.cc:1428] Setting key 46eaba2cc6760712760545187d8660ec
I20260809 16:20:37.131052 27195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:37.121543 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:33143 (local address 127.21.51.131:45127)
0809 16:20:37.121726 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0809 16:20:37.121770 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 16:20:37.122278 (+ 508us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:20:37.122552 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:20:37.122558 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:20:37.123285 (+ 727us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:20:37.123498 (+ 213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:20:37.123503 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:20:37.124219 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:20:37.124220 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:20:37.125278 (+ 1058us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:20:37.125281 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:20:37.125637 (+ 356us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:20:37.125638 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:20:37.125785 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:37.126313 (+ 528us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:20:37.126321 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:20:37.127861 (+ 1540us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:20:37.129511 (+ 1650us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:20:37.129517 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:20:37.129519 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:20:37.129802 (+ 283us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:20:37.130015 (+ 213us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:20:37.130016 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:20:37.130016 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:20:37.130131 (+ 115us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:20:37.130385 (+ 254us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:20:37.130390 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:20:37.130527 (+ 137us) client_negotiation.cc:769] Sending connection context
0809 16:20:37.130620 (+ 93us) client_negotiation.cc:240] Negotiation successful
0809 16:20:37.130803 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":99,"threads_started":1}
I20260809 16:20:37.131695 27193 heartbeater.cc:347] Connected to a master server at 127.21.51.190:33143
I20260809 16:20:37.131810 27193 heartbeater.cc:464] Registering TS with master...
I20260809 16:20:37.132004 27193 heartbeater.cc:511] Master 127.21.51.190:33143 requested a full tablet report, sending...
I20260809 16:20:37.132408 26497 ts_manager.cc:194] Registered new tserver with Master: a307d365d363472fa4138911611010ed (127.21.51.131:41841)
I20260809 16:20:37.132890 26497 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.51.131@KRBTEST.COM'} at 127.21.51.131:45127
I20260809 16:20:37.143002 21710 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 09 16:20:37 dist-test-slave-s11t krb5kdc[26425](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292437, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Aug 09 16:20:37 dist-test-slave-s11t krb5kdc[26425](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292437, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.21.51.190@KRBTEST.COM
I20260809 16:20:37.168661 26558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:37.161073 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44852 (local address 127.21.51.190:33143)
0809 16:20:37.161159 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0809 16:20:37.161161 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:20:37.161266 (+ 105us) server_negotiation.cc:407] Connection header received
0809 16:20:37.161337 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:20:37.161338 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:20:37.161347 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:20:37.161400 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:20:37.162064 (+ 664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:37.162581 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:37.163384 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:37.163574 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:37.163632 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:37.166286 (+ 2654us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:20:37.166287 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:20:37.166288 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:20:37.166295 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:20:37.167646 (+ 1351us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:20:37.167959 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:20:37.167960 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:20:37.167960 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:20:37.168014 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:20:37.168180 (+ 166us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:20:37.168180 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:20:37.168181 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:20:37.168332 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:20:37.168408 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0809 16:20:37.168526 (+ 118us) server_negotiation.cc:299] Negotiation successful
0809 16:20:37.168593 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
W20260809 16:20:37.168996 26497 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:44852
I20260809 16:20:37.169447 21710 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrNqUCn/build/release/bin/kudu with pid 26590
I20260809 16:20:37.178097 21710 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrNqUCn/build/release/bin/kudu with pid 26790
I20260809 16:20:37.185423 21710 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrNqUCn/build/release/bin/kudu with pid 26989
I20260809 16:20:37.199095 21710 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrNqUCn/build/release/bin/kudu with pid 26443
2026-08-09T16:20:37Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (2154 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786292417722232-21710-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 16:20:37 dist-test-slave-s11t krb5kdc[27207](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 16:20:37 dist-test-slave-s11t krb5kdc[27207](info): set up 2 sockets
Aug 09 16:20:37 dist-test-slave-s11t krb5kdc[27207](info): commencing operation
krb5kdc: starting...
W20260809 16:20:38.448377 21710 mini_kdc.cc:121] Time spent starting KDC: real 1.224s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786292417722232-21710-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786292417722232-21710-0/krb5kdc/test-user.keytab.
Aug 09 16:20:38 dist-test-slave-s11t krb5kdc[27207](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292438, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T16:20:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T16:20:38Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.51.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.51.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.51.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.190.keytab.
Entry for principal kudu/127.21.51.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.190.keytab.
I20260809 16:20:38.570966 21710 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrNqUCn/build/release/bin/kudu
/tmp/dist-test-taskrNqUCn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786292417722232-21710-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786292417722232-21710-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786292417722232-21710-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786292417722232-21710-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.51.190:40199
--webserver_interface=127.21.51.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.190.keytab
--principal=kudu/127.21.51.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.51.148:46087
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.51.190:40199
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786292417722232-21710-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786292417722232-21710-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786292417722232-21710-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:20:38.653375 27230 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:20:38.653625 27230 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:20:38.653663 27230 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:20:38.654808 27230 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 16:20:38.654886 27230 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:20:38.654911 27230 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:20:38.654940 27230 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 16:20:38.654970 27230 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 16:20:38.656463 27230 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.51.148:46087
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786292417722232-21710-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786292417722232-21710-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.51.190:40199
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.190.keytab
--principal=kudu/127.21.51.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.51.190:40199
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786292417722232-21710-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.51.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27230
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786292417722232-21710-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:20:38.656766 27230 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:20:38.657014 27230 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:20:38.662048 27244 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:20:38.662061 27237 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:20:38.662056 27238 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:20:38.662441 27230 server_base.cc:1034] running on GCE node
I20260809 16:20:38.662694 27230 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:20:38.662983 27230 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:20:38.664160 27230 hybrid_clock.cc:648] HybridClock initialized: now 1786292438664139 us; error 42 us; skew 500 ppm
Aug 09 16:20:38 dist-test-slave-s11t krb5kdc[27207](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292438, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.51.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:20:38.666675 27230 init.cc:377] Logged in from keytab as kudu/127.21.51.190@KRBTEST.COM (short username kudu)
I20260809 16:20:38.667163 27230 webserver.cc:460] Webserver started at http://127.21.51.190:43749/ using document root <none> and password file <none>
I20260809 16:20:38.667399 27230 fs_manager.cc:359] Metadata directory not provided
I20260809 16:20:38.667469 27230 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:20:38.667626 27230 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:20:38.668666 27230 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786292417722232-21710-0/minicluster-data/master-0/data/instance:
uuid: "350273271262492ca77c6f7fb22c9830"
format_stamp: "Formatted at 2026-08-09 16:20:38 on dist-test-slave-s11t"
server_key: "c6954083f7734964335af3e664e18d53"
server_key_iv: "ce2d9083d939ed05dd2752bc7d263863"
server_key_version: "encryptionkey@0"
I20260809 16:20:38.669111 27230 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786292417722232-21710-0/minicluster-data/master-0/wal/instance:
uuid: "350273271262492ca77c6f7fb22c9830"
format_stamp: "Formatted at 2026-08-09 16:20:38 on dist-test-slave-s11t"
server_key: "c6954083f7734964335af3e664e18d53"
server_key_iv: "ce2d9083d939ed05dd2752bc7d263863"
server_key_version: "encryptionkey@0"
I20260809 16:20:38.670557 27230 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260809 16:20:38.671296 27254 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:20:38.671476 27230 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 16:20:38.671600 27230 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786292417722232-21710-0/minicluster-data/master-0/data,/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786292417722232-21710-0/minicluster-data/master-0/wal
uuid: "350273271262492ca77c6f7fb22c9830"
format_stamp: "Formatted at 2026-08-09 16:20:38 on dist-test-slave-s11t"
server_key: "c6954083f7734964335af3e664e18d53"
server_key_iv: "ce2d9083d939ed05dd2752bc7d263863"
server_key_version: "encryptionkey@0"
I20260809 16:20:38.671679 27230 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786292417722232-21710-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786292417722232-21710-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786292417722232-21710-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:20:38.679845 27230 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:20:38.681684 27230 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:20:38.681845 27230 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:20:38.685942 27230 rpc_server.cc:307] RPC server started. Bound to: 127.21.51.190:40199
I20260809 16:20:38.685984 27336 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.51.190:40199 every 8 connection(s)
I20260809 16:20:38.686511 27230 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786292417722232-21710-0/minicluster-data/master-0/data/info.pb
I20260809 16:20:38.687211 27337 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:20:38.689716 27337 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 350273271262492ca77c6f7fb22c9830: Bootstrap starting.
I20260809 16:20:38.690366 27337 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 350273271262492ca77c6f7fb22c9830: Neither blocks nor log segments found. Creating new log.
I20260809 16:20:38.690693 27337 log.cc:824] T 00000000000000000000000000000000 P 350273271262492ca77c6f7fb22c9830: Log is configured to *not* fsync() on all Append() calls
I20260809 16:20:38.691450 27337 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 350273271262492ca77c6f7fb22c9830: No bootstrap required, opened a new log
I20260809 16:20:38.693210 27337 raft_consensus.cc:348] T 00000000000000000000000000000000 P 350273271262492ca77c6f7fb22c9830 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "350273271262492ca77c6f7fb22c9830" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 40199 } }
I20260809 16:20:38.693364 27337 raft_consensus.cc:374] T 00000000000000000000000000000000 P 350273271262492ca77c6f7fb22c9830 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:20:38.693390 27337 raft_consensus.cc:729] T 00000000000000000000000000000000 P 350273271262492ca77c6f7fb22c9830 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 350273271262492ca77c6f7fb22c9830, State: Initialized, Role: FOLLOWER
I20260809 16:20:38.693580 27337 consensus_queue.cc:260] T 00000000000000000000000000000000 P 350273271262492ca77c6f7fb22c9830 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "350273271262492ca77c6f7fb22c9830" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 40199 } }
I20260809 16:20:38.693648 27337 raft_consensus.cc:388] T 00000000000000000000000000000000 P 350273271262492ca77c6f7fb22c9830 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 16:20:38.693696 27337 raft_consensus.cc:482] T 00000000000000000000000000000000 P 350273271262492ca77c6f7fb22c9830 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 16:20:38.693748 27337 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 350273271262492ca77c6f7fb22c9830 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:20:38.694698 27337 raft_consensus.cc:504] T 00000000000000000000000000000000 P 350273271262492ca77c6f7fb22c9830 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "350273271262492ca77c6f7fb22c9830" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 40199 } }
I20260809 16:20:38.694825 27337 leader_election.cc:302] T 00000000000000000000000000000000 P 350273271262492ca77c6f7fb22c9830 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 350273271262492ca77c6f7fb22c9830; no voters:
I20260809 16:20:38.695025 27337 leader_election.cc:288] T 00000000000000000000000000000000 P 350273271262492ca77c6f7fb22c9830 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 16:20:38.695094 27340 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 350273271262492ca77c6f7fb22c9830 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:20:38.695276 27340 raft_consensus.cc:686] T 00000000000000000000000000000000 P 350273271262492ca77c6f7fb22c9830 [term 1 LEADER]: Becoming Leader. State: Replica: 350273271262492ca77c6f7fb22c9830, State: Running, Role: LEADER
I20260809 16:20:38.695317 27337 sys_catalog.cc:564] T 00000000000000000000000000000000 P 350273271262492ca77c6f7fb22c9830 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 16:20:38.695462 27340 consensus_queue.cc:237] T 00000000000000000000000000000000 P 350273271262492ca77c6f7fb22c9830 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "350273271262492ca77c6f7fb22c9830" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 40199 } }
I20260809 16:20:38.695928 27344 sys_catalog.cc:455] T 00000000000000000000000000000000 P 350273271262492ca77c6f7fb22c9830 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 350273271262492ca77c6f7fb22c9830. Latest consensus state: current_term: 1 leader_uuid: "350273271262492ca77c6f7fb22c9830" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "350273271262492ca77c6f7fb22c9830" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 40199 } } }
I20260809 16:20:38.696045 27344 sys_catalog.cc:458] T 00000000000000000000000000000000 P 350273271262492ca77c6f7fb22c9830 [sys.catalog]: This master's current role is: LEADER
I20260809 16:20:38.695906 27343 sys_catalog.cc:455] T 00000000000000000000000000000000 P 350273271262492ca77c6f7fb22c9830 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "350273271262492ca77c6f7fb22c9830" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "350273271262492ca77c6f7fb22c9830" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 40199 } } }
I20260809 16:20:38.696395 27343 sys_catalog.cc:458] T 00000000000000000000000000000000 P 350273271262492ca77c6f7fb22c9830 [sys.catalog]: This master's current role is: LEADER
I20260809 16:20:38.696441 27357 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 16:20:38.696455 21710 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrNqUCn/build/release/bin/kudu as pid 27230
I20260809 16:20:38.696544 21710 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786292417722232-21710-0/minicluster-data/master-0/wal/instance
I20260809 16:20:38.696834 21710 external_mini_cluster.cc:1428] Setting key ecbf6aa9dd59634e1970d9cc4ecba779
I20260809 16:20:38.697022 27357 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 16:20:38.698642 27357 catalog_manager.cc:1347] Generated new cluster ID: 2372e29d38c74c4c89de53c40823996e
I20260809 16:20:38.698736 27357 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 09 16:20:38 dist-test-slave-s11t krb5kdc[27207](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292438, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.51.190@KRBTEST.COM
I20260809 16:20:38.708165 27370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:38.697503 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33052 (local address 127.21.51.190:40199)
0809 16:20:38.698210 (+ 707us) server_negotiation.cc:206] Beginning negotiation
0809 16:20:38.698228 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0809 16:20:38.698299 (+ 71us) server_negotiation.cc:407] Connection header received
0809 16:20:38.698431 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:20:38.698438 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:20:38.698497 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:20:38.699331 (+ 834us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:20:38.699409 (+ 78us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:38.700412 (+ 1003us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:38.701339 (+ 927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:38.701820 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:38.701953 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:38.704477 (+ 2524us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:20:38.704481 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:20:38.704481 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:20:38.704494 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:20:38.706422 (+ 1928us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:20:38.706832 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:20:38.706835 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:20:38.706836 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:20:38.706937 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:20:38.707173 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:20:38.707174 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:20:38.707174 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:20:38.707452 (+ 278us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:20:38.707642 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0809 16:20:38.707772 (+ 130us) server_negotiation.cc:299] Negotiation successful
0809 16:20:38.707935 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":658,"thread_start_us":91,"threads_started":1}
I20260809 16:20:38.715375 27357 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 16:20:38.716069 27357 catalog_manager.cc:1504] Loading token signing keys...
I20260809 16:20:38.723124 27357 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 350273271262492ca77c6f7fb22c9830: Generated new TSK 0
I20260809 16:20:38.723300 27357 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.21.51.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.51.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.51.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.129.keytab.
Entry for principal kudu/127.21.51.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.129.keytab.
I20260809 16:20:38.753257 21710 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrNqUCn/build/release/bin/kudu
/tmp/dist-test-taskrNqUCn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786292417722232-21710-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786292417722232-21710-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786292417722232-21710-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786292417722232-21710-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.51.129:0
--local_ip_for_outbound_sockets=127.21.51.129
--webserver_interface=127.21.51.129
--webserver_port=0
--tserver_master_addrs=127.21.51.190:40199
--keytab_file=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.129.keytab
--principal=kudu/127.21.51.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.51.148:46087
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786292417722232-21710-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786292417722232-21710-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786292417722232-21710-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:20:38.849442 27376 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:20:38.849670 27376 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:20:38.849707 27376 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:20:38.850840 27376 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:20:38.850905 27376 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:20:38.850950 27376 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.51.129
I20260809 16:20:38.852468 27376 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.51.148:46087
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786292417722232-21710-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786292417722232-21710-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.129.keytab
--principal=kudu/127.21.51.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.51.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786292417722232-21710-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.51.129
--webserver_port=0
--tserver_master_addrs=127.21.51.190:40199
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27376
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.51.129
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786292417722232-21710-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:20:38.852761 27376 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:20:38.852991 27376 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:20:38.858846 27388 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:20:38.858851 27385 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:20:38.858851 27384 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:20:38.859319 27376 server_base.cc:1034] running on GCE node
I20260809 16:20:38.859568 27376 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:20:38.859808 27376 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:20:38.860965 27376 hybrid_clock.cc:648] HybridClock initialized: now 1786292438860950 us; error 32 us; skew 500 ppm
Aug 09 16:20:38 dist-test-slave-s11t krb5kdc[27207](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292438, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.51.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:20:38.863543 27376 init.cc:377] Logged in from keytab as kudu/127.21.51.129@KRBTEST.COM (short username kudu)
I20260809 16:20:38.864027 27376 webserver.cc:460] Webserver started at http://127.21.51.129:36471/ using document root <none> and password file <none>
I20260809 16:20:38.864256 27376 fs_manager.cc:359] Metadata directory not provided
I20260809 16:20:38.864326 27376 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:20:38.864483 27376 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:20:38.865597 27376 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786292417722232-21710-0/minicluster-data/ts-0/data/instance:
uuid: "bcedaccfde324f9a8316ffbdf11d78cf"
format_stamp: "Formatted at 2026-08-09 16:20:38 on dist-test-slave-s11t"
server_key: "9fd955dd80e745e42726c0ff32951611"
server_key_iv: "d34a2890ac819d34dfa7d532b6e39a75"
server_key_version: "encryptionkey@0"
I20260809 16:20:38.865970 27376 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786292417722232-21710-0/minicluster-data/ts-0/wal/instance:
uuid: "bcedaccfde324f9a8316ffbdf11d78cf"
format_stamp: "Formatted at 2026-08-09 16:20:38 on dist-test-slave-s11t"
server_key: "9fd955dd80e745e42726c0ff32951611"
server_key_iv: "d34a2890ac819d34dfa7d532b6e39a75"
server_key_version: "encryptionkey@0"
I20260809 16:20:38.867398 27376 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 16:20:38.868197 27397 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:20:38.868412 27376 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:20:38.868512 27376 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786292417722232-21710-0/minicluster-data/ts-0/data,/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786292417722232-21710-0/minicluster-data/ts-0/wal
uuid: "bcedaccfde324f9a8316ffbdf11d78cf"
format_stamp: "Formatted at 2026-08-09 16:20:38 on dist-test-slave-s11t"
server_key: "9fd955dd80e745e42726c0ff32951611"
server_key_iv: "d34a2890ac819d34dfa7d532b6e39a75"
server_key_version: "encryptionkey@0"
I20260809 16:20:38.868595 27376 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786292417722232-21710-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786292417722232-21710-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786292417722232-21710-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:20:38.887027 27376 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:20:38.889380 27376 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:20:38.889550 27376 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:20:38.889796 27376 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:20:38.890116 27376 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:20:38.890170 27376 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:20:38.890208 27376 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:20:38.890236 27376 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:20:38.896394 27376 rpc_server.cc:307] RPC server started. Bound to: 127.21.51.129:42953
I20260809 16:20:38.896494 27575 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.51.129:42953 every 8 connection(s)
I20260809 16:20:38.896834 27376 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786292417722232-21710-0/minicluster-data/ts-0/data/info.pb
I20260809 16:20:38.898587 21710 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrNqUCn/build/release/bin/kudu as pid 27376
I20260809 16:20:38.898669 21710 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786292417722232-21710-0/minicluster-data/ts-0/wal/instance
I20260809 16:20:38.898883 21710 external_mini_cluster.cc:1428] Setting key b5f37ff7aacd6fce0d0cead518bf3c3b
Aug 09 16:20:38 dist-test-slave-s11t krb5kdc[27207](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292438, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.51.129@KRBTEST.COM for kudu/127.21.51.190@KRBTEST.COM
I20260809 16:20:38.907413 27370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:38.897639 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.51.129:60751 (local address 127.21.51.190:40199)
0809 16:20:38.897685 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0809 16:20:38.897687 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:20:38.897879 (+ 192us) server_negotiation.cc:407] Connection header received
0809 16:20:38.898879 (+ 1000us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:20:38.898881 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:20:38.898897 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:20:38.898963 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:20:38.899688 (+ 725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:38.900443 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:38.901759 (+ 1316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:38.901967 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:38.902022 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:38.904446 (+ 2424us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:20:38.904448 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:20:38.904448 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:20:38.904458 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:20:38.905952 (+ 1494us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:20:38.906405 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:20:38.906406 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:20:38.906407 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:20:38.906471 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:20:38.906732 (+ 261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:20:38.906733 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:20:38.906733 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:20:38.906858 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:20:38.906933 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0809 16:20:38.907219 (+ 286us) server_negotiation.cc:299] Negotiation successful
0809 16:20:38.907321 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260809 16:20:38.907583 27580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:38.897599 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:40199 (local address 127.21.51.129:60751)
0809 16:20:38.897751 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0809 16:20:38.897792 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0809 16:20:38.898747 (+ 955us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:20:38.899044 (+ 297us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:20:38.899049 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:20:38.899337 (+ 288us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:20:38.899586 (+ 249us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:20:38.899592 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:20:38.900533 (+ 941us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:20:38.900534 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:20:38.901658 (+ 1124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:20:38.901661 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:20:38.902054 (+ 393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:20:38.902055 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:20:38.902200 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:38.902707 (+ 507us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:20:38.902718 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:20:38.904336 (+ 1618us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:20:38.906059 (+ 1723us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:20:38.906067 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:20:38.906069 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:20:38.906309 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:20:38.906541 (+ 232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:20:38.906541 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:20:38.906542 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:20:38.906659 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:20:38.906936 (+ 277us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:20:38.906944 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:20:38.907080 (+ 136us) client_negotiation.cc:769] Sending connection context
0809 16:20:38.907182 (+ 102us) client_negotiation.cc:240] Negotiation successful
0809 16:20:38.907346 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":82,"threads_started":1}
WARNING: no policy specified for kudu/127.21.51.130@KRBTEST.COM; defaulting to no policy
I20260809 16:20:38.908118 27578 heartbeater.cc:347] Connected to a master server at 127.21.51.190:40199
I20260809 16:20:38.908201 27578 heartbeater.cc:464] Registering TS with master...
I20260809 16:20:38.908339 27578 heartbeater.cc:511] Master 127.21.51.190:40199 requested a full tablet report, sending...
I20260809 16:20:38.908910 27280 ts_manager.cc:194] Registered new tserver with Master: bcedaccfde324f9a8316ffbdf11d78cf (127.21.51.129:42953)
I20260809 16:20:38.909520 27280 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.51.129@KRBTEST.COM'} at 127.21.51.129:60751
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.51.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.51.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.130.keytab.
Entry for principal kudu/127.21.51.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.130.keytab.
I20260809 16:20:38.924822 21710 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrNqUCn/build/release/bin/kudu
/tmp/dist-test-taskrNqUCn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786292417722232-21710-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786292417722232-21710-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786292417722232-21710-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786292417722232-21710-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.51.130:0
--local_ip_for_outbound_sockets=127.21.51.130
--webserver_interface=127.21.51.130
--webserver_port=0
--tserver_master_addrs=127.21.51.190:40199
--keytab_file=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.130.keytab
--principal=kudu/127.21.51.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.51.148:46087
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786292417722232-21710-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786292417722232-21710-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786292417722232-21710-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:20:39.006705 27584 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:20:39.006922 27584 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:20:39.006946 27584 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:20:39.008016 27584 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:20:39.008049 27584 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:20:39.008074 27584 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.51.130
I20260809 16:20:39.009662 27584 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.51.148:46087
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786292417722232-21710-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786292417722232-21710-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.130.keytab
--principal=kudu/127.21.51.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.51.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786292417722232-21710-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.51.130
--webserver_port=0
--tserver_master_addrs=127.21.51.190:40199
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27584
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.51.130
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786292417722232-21710-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:20:39.009846 27584 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:20:39.010026 27584 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:20:39.015154 27594 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:20:39.015228 27584 server_base.cc:1034] running on GCE node
W20260809 16:20:39.015154 27591 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:20:39.015163 27590 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:20:39.015626 27584 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:20:39.015911 27584 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:20:39.017081 27584 hybrid_clock.cc:648] HybridClock initialized: now 1786292439017074 us; error 30 us; skew 500 ppm
Aug 09 16:20:39 dist-test-slave-s11t krb5kdc[27207](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292439, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.51.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:20:39.019840 27584 init.cc:377] Logged in from keytab as kudu/127.21.51.130@KRBTEST.COM (short username kudu)
I20260809 16:20:39.020314 27584 webserver.cc:460] Webserver started at http://127.21.51.130:34619/ using document root <none> and password file <none>
I20260809 16:20:39.020535 27584 fs_manager.cc:359] Metadata directory not provided
I20260809 16:20:39.020581 27584 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:20:39.020772 27584 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:20:39.021817 27584 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786292417722232-21710-0/minicluster-data/ts-1/data/instance:
uuid: "f1f8e88ef0f045f6a03d9ba18033eabc"
format_stamp: "Formatted at 2026-08-09 16:20:39 on dist-test-slave-s11t"
server_key: "bf99d6aac8de77ff872d35570f0ccf32"
server_key_iv: "696b93ea1029b9af27a09c57cc07295e"
server_key_version: "encryptionkey@0"
I20260809 16:20:39.022176 27584 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786292417722232-21710-0/minicluster-data/ts-1/wal/instance:
uuid: "f1f8e88ef0f045f6a03d9ba18033eabc"
format_stamp: "Formatted at 2026-08-09 16:20:39 on dist-test-slave-s11t"
server_key: "bf99d6aac8de77ff872d35570f0ccf32"
server_key_iv: "696b93ea1029b9af27a09c57cc07295e"
server_key_version: "encryptionkey@0"
I20260809 16:20:39.023576 27584 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 16:20:39.024494 27612 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:20:39.024801 27584 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:20:39.024916 27584 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786292417722232-21710-0/minicluster-data/ts-1/data,/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786292417722232-21710-0/minicluster-data/ts-1/wal
uuid: "f1f8e88ef0f045f6a03d9ba18033eabc"
format_stamp: "Formatted at 2026-08-09 16:20:39 on dist-test-slave-s11t"
server_key: "bf99d6aac8de77ff872d35570f0ccf32"
server_key_iv: "696b93ea1029b9af27a09c57cc07295e"
server_key_version: "encryptionkey@0"
I20260809 16:20:39.025003 27584 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786292417722232-21710-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786292417722232-21710-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786292417722232-21710-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:20:39.040079 27584 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:20:39.042744 27584 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:20:39.042924 27584 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:20:39.043267 27584 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:20:39.043637 27584 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:20:39.043681 27584 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:20:39.043707 27584 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:20:39.043722 27584 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:20:39.050413 27584 rpc_server.cc:307] RPC server started. Bound to: 127.21.51.130:39491
I20260809 16:20:39.050519 27791 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.51.130:39491 every 8 connection(s)
I20260809 16:20:39.050843 27584 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786292417722232-21710-0/minicluster-data/ts-1/data/info.pb
Aug 09 16:20:39 dist-test-slave-s11t krb5kdc[27207](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292439, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.51.130@KRBTEST.COM for kudu/127.21.51.190@KRBTEST.COM
I20260809 16:20:39.059993 21710 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrNqUCn/build/release/bin/kudu as pid 27584
I20260809 16:20:39.060089 21710 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786292417722232-21710-0/minicluster-data/ts-1/wal/instance
I20260809 16:20:39.060278 21710 external_mini_cluster.cc:1428] Setting key 95b3fc80e2f45dd5ad071f7d2526e518
I20260809 16:20:39.060745 27370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:39.051656 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.51.130:35677 (local address 127.21.51.190:40199)
0809 16:20:39.051712 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0809 16:20:39.051714 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:20:39.051932 (+ 218us) server_negotiation.cc:407] Connection header received
0809 16:20:39.052861 (+ 929us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:20:39.052862 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:20:39.052873 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:20:39.052938 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:20:39.053532 (+ 594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:39.054045 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:39.055156 (+ 1111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:39.055360 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:39.055417 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:39.057527 (+ 2110us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:20:39.057529 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:20:39.057529 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:20:39.057535 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:20:39.059000 (+ 1465us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:20:39.059344 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:20:39.059345 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:20:39.059345 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:20:39.059398 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:20:39.059690 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:20:39.059690 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:20:39.059691 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:20:39.059890 (+ 199us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:20:39.059968 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0809 16:20:39.060447 (+ 479us) server_negotiation.cc:299] Negotiation successful
0809 16:20:39.060592 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260809 16:20:39.060864 27796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:39.051608 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:40199 (local address 127.21.51.130:35677)
0809 16:20:39.051780 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0809 16:20:39.051823 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0809 16:20:39.052771 (+ 948us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:20:39.052993 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:20:39.052996 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:20:39.053255 (+ 259us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:20:39.053448 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:20:39.053452 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:20:39.054126 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:20:39.054127 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:20:39.055069 (+ 942us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:20:39.055072 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:20:39.055437 (+ 365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:20:39.055438 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:20:39.055535 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:39.056020 (+ 485us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:20:39.056028 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:20:39.057445 (+ 1417us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:20:39.059067 (+ 1622us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:20:39.059072 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:20:39.059073 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:20:39.059278 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:20:39.059457 (+ 179us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:20:39.059457 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:20:39.059458 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:20:39.059609 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:20:39.059993 (+ 384us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:20:39.060001 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:20:39.060254 (+ 253us) client_negotiation.cc:769] Sending connection context
0809 16:20:39.060389 (+ 135us) client_negotiation.cc:240] Negotiation successful
0809 16:20:39.060585 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":91,"threads_started":1}
I20260809 16:20:39.061344 27793 heartbeater.cc:347] Connected to a master server at 127.21.51.190:40199
I20260809 16:20:39.061450 27793 heartbeater.cc:464] Registering TS with master...
I20260809 16:20:39.061638 27793 heartbeater.cc:511] Master 127.21.51.190:40199 requested a full tablet report, sending...
I20260809 16:20:39.061978 27280 ts_manager.cc:194] Registered new tserver with Master: f1f8e88ef0f045f6a03d9ba18033eabc (127.21.51.130:39491)
I20260809 16:20:39.062383 27280 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.51.130@KRBTEST.COM'} at 127.21.51.130:35677
WARNING: no policy specified for kudu/127.21.51.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.51.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.51.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.131.keytab.
Entry for principal kudu/127.21.51.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.131.keytab.
I20260809 16:20:39.087363 21710 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrNqUCn/build/release/bin/kudu
/tmp/dist-test-taskrNqUCn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786292417722232-21710-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786292417722232-21710-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786292417722232-21710-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786292417722232-21710-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.51.131:0
--local_ip_for_outbound_sockets=127.21.51.131
--webserver_interface=127.21.51.131
--webserver_port=0
--tserver_master_addrs=127.21.51.190:40199
--keytab_file=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.131.keytab
--principal=kudu/127.21.51.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.51.148:46087
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786292417722232-21710-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786292417722232-21710-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786292417722232-21710-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:20:39.170809 27799 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:20:39.171013 27799 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:20:39.171036 27799 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:20:39.172312 27799 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:20:39.172354 27799 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:20:39.172380 27799 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.51.131
I20260809 16:20:39.173884 27799 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.51.148:46087
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786292417722232-21710-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786292417722232-21710-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.131.keytab
--principal=kudu/127.21.51.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.51.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786292417722232-21710-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.51.131
--webserver_port=0
--tserver_master_addrs=127.21.51.190:40199
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27799
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.51.131
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786292417722232-21710-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:20:39.174125 27799 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:20:39.174381 27799 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:20:39.179498 27812 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:20:39.179522 27809 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:20:39.179529 27810 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:20:39.179734 27799 server_base.cc:1034] running on GCE node
I20260809 16:20:39.180025 27799 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:20:39.180259 27799 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:20:39.181423 27799 hybrid_clock.cc:648] HybridClock initialized: now 1786292439181420 us; error 47 us; skew 500 ppm
Aug 09 16:20:39 dist-test-slave-s11t krb5kdc[27207](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292439, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.51.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:20:39.184067 27799 init.cc:377] Logged in from keytab as kudu/127.21.51.131@KRBTEST.COM (short username kudu)
I20260809 16:20:39.184547 27799 webserver.cc:460] Webserver started at http://127.21.51.131:33519/ using document root <none> and password file <none>
I20260809 16:20:39.184827 27799 fs_manager.cc:359] Metadata directory not provided
I20260809 16:20:39.184897 27799 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:20:39.185065 27799 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:20:39.186093 27799 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786292417722232-21710-0/minicluster-data/ts-2/data/instance:
uuid: "96ce84c97482435f97c1340f00894820"
format_stamp: "Formatted at 2026-08-09 16:20:39 on dist-test-slave-s11t"
server_key: "63490c9d7b897120a6e5648b6e81796e"
server_key_iv: "6d0a574963f67473cdfb893517fa56a5"
server_key_version: "encryptionkey@0"
I20260809 16:20:39.186461 27799 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786292417722232-21710-0/minicluster-data/ts-2/wal/instance:
uuid: "96ce84c97482435f97c1340f00894820"
format_stamp: "Formatted at 2026-08-09 16:20:39 on dist-test-slave-s11t"
server_key: "63490c9d7b897120a6e5648b6e81796e"
server_key_iv: "6d0a574963f67473cdfb893517fa56a5"
server_key_version: "encryptionkey@0"
I20260809 16:20:39.187774 27799 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 16:20:39.188549 27822 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:20:39.188766 27799 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:20:39.188866 27799 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786292417722232-21710-0/minicluster-data/ts-2/data,/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786292417722232-21710-0/minicluster-data/ts-2/wal
uuid: "96ce84c97482435f97c1340f00894820"
format_stamp: "Formatted at 2026-08-09 16:20:39 on dist-test-slave-s11t"
server_key: "63490c9d7b897120a6e5648b6e81796e"
server_key_iv: "6d0a574963f67473cdfb893517fa56a5"
server_key_version: "encryptionkey@0"
I20260809 16:20:39.188963 27799 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786292417722232-21710-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786292417722232-21710-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786292417722232-21710-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:20:39.201615 27799 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:20:39.203796 27799 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:20:39.203976 27799 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:20:39.204219 27799 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:20:39.204535 27799 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:20:39.204591 27799 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:20:39.204634 27799 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:20:39.204659 27799 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:20:39.210882 27799 rpc_server.cc:307] RPC server started. Bound to: 127.21.51.131:34629
I20260809 16:20:39.210937 28007 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.51.131:34629 every 8 connection(s)
I20260809 16:20:39.211277 27799 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786292417722232-21710-0/minicluster-data/ts-2/data/info.pb
I20260809 16:20:39.212496 21710 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrNqUCn/build/release/bin/kudu as pid 27799
I20260809 16:20:39.212605 21710 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786292417722232-21710-0/minicluster-data/ts-2/wal/instance
I20260809 16:20:39.212817 21710 external_mini_cluster.cc:1428] Setting key 496326b751a35b0a8ccf4ea144ab5344
Aug 09 16:20:39 dist-test-slave-s11t krb5kdc[27207](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292439, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.51.131@KRBTEST.COM for kudu/127.21.51.190@KRBTEST.COM
I20260809 16:20:39.221662 27370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:39.212073 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.51.131:49763 (local address 127.21.51.190:40199)
0809 16:20:39.212159 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0809 16:20:39.212162 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 16:20:39.212301 (+ 139us) server_negotiation.cc:407] Connection header received
0809 16:20:39.213582 (+ 1281us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:20:39.213583 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:20:39.213594 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:20:39.213659 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:20:39.214447 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:39.214989 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:39.216122 (+ 1133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:39.216324 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:39.216390 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:39.218724 (+ 2334us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:20:39.218726 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:20:39.218726 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:20:39.218734 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:20:39.220147 (+ 1413us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:20:39.220621 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:20:39.220622 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:20:39.220623 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:20:39.220747 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:20:39.221006 (+ 259us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:20:39.221007 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:20:39.221008 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:20:39.221125 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:20:39.221181 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0809 16:20:39.221517 (+ 336us) server_negotiation.cc:299] Negotiation successful
0809 16:20:39.221599 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260809 16:20:39.221799 28013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:39.211997 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:40199 (local address 127.21.51.131:49763)
0809 16:20:39.212159 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0809 16:20:39.212200 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0809 16:20:39.213451 (+ 1251us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:20:39.213742 (+ 291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:20:39.213747 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:20:39.214115 (+ 368us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:20:39.214336 (+ 221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:20:39.214342 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:20:39.215109 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:20:39.215110 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:20:39.216026 (+ 916us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:20:39.216028 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:20:39.216427 (+ 399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:20:39.216429 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:20:39.216543 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:39.217191 (+ 648us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:20:39.217218 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:20:39.218601 (+ 1383us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:20:39.220255 (+ 1654us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:20:39.220261 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:20:39.220262 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:20:39.220523 (+ 261us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:20:39.220831 (+ 308us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:20:39.220832 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:20:39.220832 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:20:39.220946 (+ 114us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:20:39.221210 (+ 264us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:20:39.221215 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:20:39.221350 (+ 135us) client_negotiation.cc:769] Sending connection context
0809 16:20:39.221442 (+ 92us) client_negotiation.cc:240] Negotiation successful
0809 16:20:39.221574 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":69,"threads_started":1}
I20260809 16:20:39.222323 28008 heartbeater.cc:347] Connected to a master server at 127.21.51.190:40199
I20260809 16:20:39.222419 28008 heartbeater.cc:464] Registering TS with master...
I20260809 16:20:39.222601 28008 heartbeater.cc:511] Master 127.21.51.190:40199 requested a full tablet report, sending...
I20260809 16:20:39.223057 27280 ts_manager.cc:194] Registered new tserver with Master: 96ce84c97482435f97c1340f00894820 (127.21.51.131:34629)
I20260809 16:20:39.223493 27280 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.51.131@KRBTEST.COM'} at 127.21.51.131:49763
I20260809 16:20:39.224958 21710 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 09 16:20:39 dist-test-slave-s11t krb5kdc[27207](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292439, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 09 16:20:39 dist-test-slave-s11t krb5kdc[27207](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292439, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.51.129@KRBTEST.COM
I20260809 16:20:39.253854 28021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:39.243497 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46120 (local address 127.21.51.129:42953)
0809 16:20:39.243666 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0809 16:20:39.243671 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 16:20:39.243680 (+ 9us) server_negotiation.cc:407] Connection header received
0809 16:20:39.243706 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:20:39.243708 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:20:39.243722 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:20:39.243817 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:20:39.244473 (+ 656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:39.245534 (+ 1061us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:39.246688 (+ 1154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:39.247272 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:39.247363 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:39.250585 (+ 3222us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:20:39.250594 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:20:39.250595 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:20:39.250603 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:20:39.252475 (+ 1872us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:20:39.252918 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:20:39.252921 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:20:39.252921 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:20:39.253025 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:20:39.253230 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:20:39.253231 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:20:39.253231 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:20:39.253376 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:20:39.253477 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0809 16:20:39.253607 (+ 130us) server_negotiation.cc:299] Negotiation successful
0809 16:20:39.253711 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":127,"thread_start_us":66,"threads_started":1}
Aug 09 16:20:39 dist-test-slave-s11t krb5kdc[27207](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292439, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.51.190@KRBTEST.COM
I20260809 16:20:39.264534 27370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:39.255465 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33064 (local address 127.21.51.190:40199)
0809 16:20:39.255523 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0809 16:20:39.255525 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:20:39.255533 (+ 8us) server_negotiation.cc:407] Connection header received
0809 16:20:39.255557 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:20:39.255558 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:20:39.255568 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:20:39.255624 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:20:39.256319 (+ 695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:39.257189 (+ 870us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:39.258324 (+ 1135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:39.258608 (+ 284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:39.258700 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:39.261612 (+ 2912us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:20:39.261613 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:20:39.261613 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:20:39.261620 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:20:39.263128 (+ 1508us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:20:39.263525 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:20:39.263526 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:20:39.263527 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:20:39.263584 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:20:39.263815 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:20:39.263816 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:20:39.263816 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:20:39.263989 (+ 173us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:20:39.264072 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0809 16:20:39.264283 (+ 211us) server_negotiation.cc:299] Negotiation successful
0809 16:20:39.264395 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
W20260809 16:20:39.264809 27280 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33064
I20260809 16:20:39.265362 21710 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrNqUCn/build/release/bin/kudu with pid 27376
I20260809 16:20:39.278033 21710 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrNqUCn/build/release/bin/kudu with pid 27584
I20260809 16:20:39.292995 21710 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrNqUCn/build/release/bin/kudu with pid 27799
I20260809 16:20:39.302340 21710 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrNqUCn/build/release/bin/kudu with pid 27230
2026-08-09T16:20:39Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (2104 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786292417722232-21710-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 16:20:39 dist-test-slave-s11t krb5kdc[28030](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 16:20:39 dist-test-slave-s11t krb5kdc[28030](info): set up 2 sockets
Aug 09 16:20:39 dist-test-slave-s11t krb5kdc[28030](info): commencing operation
krb5kdc: starting...
W20260809 16:20:40.540102 21710 mini_kdc.cc:121] Time spent starting KDC: real 1.212s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786292417722232-21710-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786292417722232-21710-0/krb5kdc/test-user.keytab.
Aug 09 16:20:40 dist-test-slave-s11t krb5kdc[28030](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292440, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T16:20:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T16:20:40Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.51.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.51.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.51.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.190.keytab.
Entry for principal kudu/127.21.51.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.190.keytab.
I20260809 16:20:40.661012 21710 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrNqUCn/build/release/bin/kudu
/tmp/dist-test-taskrNqUCn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786292417722232-21710-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786292417722232-21710-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786292417722232-21710-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786292417722232-21710-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.51.190:42777
--webserver_interface=127.21.51.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.190.keytab
--principal=kudu/127.21.51.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.51.148:43673
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.51.190:42777
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786292417722232-21710-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786292417722232-21710-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786292417722232-21710-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:20:40.753017 28058 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:20:40.753252 28058 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:20:40.753289 28058 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:20:40.754392 28058 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 16:20:40.754459 28058 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:20:40.754482 28058 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:20:40.754503 28058 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 16:20:40.754534 28058 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 16:20:40.756000 28058 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.51.148:43673
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786292417722232-21710-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786292417722232-21710-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.51.190:42777
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.190.keytab
--principal=kudu/127.21.51.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.51.190:42777
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786292417722232-21710-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.51.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28058
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786292417722232-21710-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:20:40.756276 28058 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:20:40.756507 28058 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:20:40.761390 28066 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:20:40.761410 28067 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:20:40.761386 28071 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:20:40.761613 28058 server_base.cc:1034] running on GCE node
I20260809 16:20:40.761842 28058 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:20:40.762094 28058 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:20:40.763249 28058 hybrid_clock.cc:648] HybridClock initialized: now 1786292440763236 us; error 30 us; skew 500 ppm
Aug 09 16:20:40 dist-test-slave-s11t krb5kdc[28030](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292440, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.51.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:20:40.769435 28058 init.cc:377] Logged in from keytab as kudu/127.21.51.190@KRBTEST.COM (short username kudu)
I20260809 16:20:40.769820 28058 fs_manager.cc:359] Metadata directory not provided
I20260809 16:20:40.769872 28058 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:20:40.770048 28058 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:20:40.771114 28058 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786292417722232-21710-0/minicluster-data/master-0/data/instance:
uuid: "0d8d83ab4e6f4c539e1291d6fd2d09e3"
format_stamp: "Formatted at 2026-08-09 16:20:40 on dist-test-slave-s11t"
server_key: "b2303e5544a34f09e617e16fbd7afaaa"
server_key_iv: "c989f90d3c938cf6023d3b69bfb6fe29"
server_key_version: "encryptionkey@0"
I20260809 16:20:40.771485 28058 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786292417722232-21710-0/minicluster-data/master-0/wal/instance:
uuid: "0d8d83ab4e6f4c539e1291d6fd2d09e3"
format_stamp: "Formatted at 2026-08-09 16:20:40 on dist-test-slave-s11t"
server_key: "b2303e5544a34f09e617e16fbd7afaaa"
server_key_iv: "c989f90d3c938cf6023d3b69bfb6fe29"
server_key_version: "encryptionkey@0"
I20260809 16:20:40.772907 28058 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260809 16:20:40.773669 28080 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:20:40.773931 28058 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:20:40.774031 28058 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786292417722232-21710-0/minicluster-data/master-0/data,/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786292417722232-21710-0/minicluster-data/master-0/wal
uuid: "0d8d83ab4e6f4c539e1291d6fd2d09e3"
format_stamp: "Formatted at 2026-08-09 16:20:40 on dist-test-slave-s11t"
server_key: "b2303e5544a34f09e617e16fbd7afaaa"
server_key_iv: "c989f90d3c938cf6023d3b69bfb6fe29"
server_key_version: "encryptionkey@0"
I20260809 16:20:40.774106 28058 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786292417722232-21710-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786292417722232-21710-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786292417722232-21710-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:20:40.787937 28058 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:20:40.789968 28058 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:20:40.790110 28058 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:20:40.794562 28058 rpc_server.cc:307] RPC server started. Bound to: 127.21.51.190:42777
I20260809 16:20:40.794587 28152 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.51.190:42777 every 8 connection(s)
I20260809 16:20:40.794899 28058 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786292417722232-21710-0/minicluster-data/master-0/data/info.pb
I20260809 16:20:40.796053 28155 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:20:40.796119 21710 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrNqUCn/build/release/bin/kudu as pid 28058
I20260809 16:20:40.796247 21710 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786292417722232-21710-0/minicluster-data/master-0/wal/instance
I20260809 16:20:40.796451 21710 external_mini_cluster.cc:1428] Setting key 981a147f6e896523cc3dcb459750d080
I20260809 16:20:40.798717 28155 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0d8d83ab4e6f4c539e1291d6fd2d09e3: Bootstrap starting.
I20260809 16:20:40.799297 28155 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0d8d83ab4e6f4c539e1291d6fd2d09e3: Neither blocks nor log segments found. Creating new log.
I20260809 16:20:40.799606 28155 log.cc:824] T 00000000000000000000000000000000 P 0d8d83ab4e6f4c539e1291d6fd2d09e3: Log is configured to *not* fsync() on all Append() calls
I20260809 16:20:40.800395 28155 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0d8d83ab4e6f4c539e1291d6fd2d09e3: No bootstrap required, opened a new log
I20260809 16:20:40.802663 28155 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0d8d83ab4e6f4c539e1291d6fd2d09e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d8d83ab4e6f4c539e1291d6fd2d09e3" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 42777 } }
I20260809 16:20:40.802852 28155 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0d8d83ab4e6f4c539e1291d6fd2d09e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:20:40.802897 28155 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0d8d83ab4e6f4c539e1291d6fd2d09e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d8d83ab4e6f4c539e1291d6fd2d09e3, State: Initialized, Role: FOLLOWER
I20260809 16:20:40.803036 28155 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0d8d83ab4e6f4c539e1291d6fd2d09e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d8d83ab4e6f4c539e1291d6fd2d09e3" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 42777 } }
I20260809 16:20:40.803098 28155 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0d8d83ab4e6f4c539e1291d6fd2d09e3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 16:20:40.803119 28155 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0d8d83ab4e6f4c539e1291d6fd2d09e3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 16:20:40.803187 28155 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0d8d83ab4e6f4c539e1291d6fd2d09e3 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:20:40.803781 28155 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0d8d83ab4e6f4c539e1291d6fd2d09e3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d8d83ab4e6f4c539e1291d6fd2d09e3" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 42777 } }
I20260809 16:20:40.803882 28155 leader_election.cc:302] T 00000000000000000000000000000000 P 0d8d83ab4e6f4c539e1291d6fd2d09e3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0d8d83ab4e6f4c539e1291d6fd2d09e3; no voters:
I20260809 16:20:40.804029 28155 leader_election.cc:288] T 00000000000000000000000000000000 P 0d8d83ab4e6f4c539e1291d6fd2d09e3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 16:20:40.804095 28161 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0d8d83ab4e6f4c539e1291d6fd2d09e3 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:20:40.804241 28161 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0d8d83ab4e6f4c539e1291d6fd2d09e3 [term 1 LEADER]: Becoming Leader. State: Replica: 0d8d83ab4e6f4c539e1291d6fd2d09e3, State: Running, Role: LEADER
I20260809 16:20:40.804276 28155 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0d8d83ab4e6f4c539e1291d6fd2d09e3 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 16:20:40.804400 28161 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0d8d83ab4e6f4c539e1291d6fd2d09e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d8d83ab4e6f4c539e1291d6fd2d09e3" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 42777 } }
I20260809 16:20:40.804780 28163 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0d8d83ab4e6f4c539e1291d6fd2d09e3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0d8d83ab4e6f4c539e1291d6fd2d09e3. Latest consensus state: current_term: 1 leader_uuid: "0d8d83ab4e6f4c539e1291d6fd2d09e3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d8d83ab4e6f4c539e1291d6fd2d09e3" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 42777 } } }
I20260809 16:20:40.804791 28164 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0d8d83ab4e6f4c539e1291d6fd2d09e3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0d8d83ab4e6f4c539e1291d6fd2d09e3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d8d83ab4e6f4c539e1291d6fd2d09e3" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 42777 } } }
I20260809 16:20:40.804913 28163 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0d8d83ab4e6f4c539e1291d6fd2d09e3 [sys.catalog]: This master's current role is: LEADER
I20260809 16:20:40.804949 28164 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0d8d83ab4e6f4c539e1291d6fd2d09e3 [sys.catalog]: This master's current role is: LEADER
I20260809 16:20:40.805146 28177 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 16:20:40.805656 28177 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 09 16:20:40 dist-test-slave-s11t krb5kdc[28030](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292440, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.51.190@KRBTEST.COM
I20260809 16:20:40.811062 28158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:40.797054 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41156 (local address 127.21.51.190:42777)
0809 16:20:40.799875 (+ 2821us) server_negotiation.cc:206] Beginning negotiation
0809 16:20:40.799884 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 16:20:40.799919 (+ 35us) server_negotiation.cc:407] Connection header received
0809 16:20:40.800455 (+ 536us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:20:40.800463 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:20:40.800531 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:20:40.800714 (+ 183us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:20:40.801315 (+ 601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:40.802784 (+ 1469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:40.803726 (+ 942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:40.804355 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:40.804843 (+ 488us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:40.807777 (+ 2934us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:20:40.807785 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:20:40.807786 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:20:40.807796 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:20:40.809487 (+ 1691us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:20:40.809889 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:20:40.809892 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:20:40.809892 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:20:40.809981 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:20:40.810209 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:20:40.810210 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:20:40.810210 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:20:40.810453 (+ 243us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:20:40.810610 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0809 16:20:40.810716 (+ 106us) server_negotiation.cc:299] Negotiation successful
0809 16:20:40.810852 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2784,"thread_start_us":85,"threads_started":1}
I20260809 16:20:40.812394 28177 catalog_manager.cc:1347] Generated new cluster ID: 8ff48948be4d4c12b6e0177f3e54afd5
I20260809 16:20:40.812453 28177 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 16:20:40.819864 28177 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 16:20:40.820565 28177 catalog_manager.cc:1504] Loading token signing keys...
I20260809 16:20:40.829773 28177 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0d8d83ab4e6f4c539e1291d6fd2d09e3: Generated new TSK 0
I20260809 16:20:40.829938 28177 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.21.51.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.51.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.51.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.129.keytab.
Entry for principal kudu/127.21.51.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.129.keytab.
I20260809 16:20:40.861819 21710 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrNqUCn/build/release/bin/kudu
/tmp/dist-test-taskrNqUCn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786292417722232-21710-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786292417722232-21710-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786292417722232-21710-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786292417722232-21710-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.51.129:0
--local_ip_for_outbound_sockets=127.21.51.129
--webserver_interface=127.21.51.129
--webserver_port=0
--tserver_master_addrs=127.21.51.190:42777
--keytab_file=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.129.keytab
--principal=kudu/127.21.51.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.51.148:43673
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786292417722232-21710-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786292417722232-21710-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786292417722232-21710-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:20:40.945109 28199 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:20:40.945313 28199 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:20:40.945338 28199 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:20:40.946493 28199 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:20:40.946537 28199 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:20:40.946563 28199 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.51.129
I20260809 16:20:40.948055 28199 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.51.148:43673
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786292417722232-21710-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786292417722232-21710-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.129.keytab
--principal=kudu/127.21.51.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.51.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786292417722232-21710-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.51.129
--webserver_port=0
--tserver_master_addrs=127.21.51.190:42777
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28199
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.51.129
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786292417722232-21710-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:20:40.948302 28199 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:20:40.948535 28199 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:20:40.953585 28213 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:20:40.953712 28199 server_base.cc:1034] running on GCE node
W20260809 16:20:40.953596 28215 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:20:40.953593 28209 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:20:40.954124 28199 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:20:40.954342 28199 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:20:40.955556 28199 hybrid_clock.cc:648] HybridClock initialized: now 1786292440955545 us; error 44 us; skew 500 ppm
Aug 09 16:20:40 dist-test-slave-s11t krb5kdc[28030](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292440, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.51.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:20:40.958227 28199 init.cc:377] Logged in from keytab as kudu/127.21.51.129@KRBTEST.COM (short username kudu)
I20260809 16:20:40.958580 28199 fs_manager.cc:359] Metadata directory not provided
I20260809 16:20:40.958660 28199 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:20:40.958829 28199 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:20:40.959928 28199 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786292417722232-21710-0/minicluster-data/ts-0/data/instance:
uuid: "896c9bb9bb2a48599f9e66bda347b48c"
format_stamp: "Formatted at 2026-08-09 16:20:40 on dist-test-slave-s11t"
server_key: "41a24985b70f8e6b12f5c40c37a5491e"
server_key_iv: "618dab347a7e2d034b5f48fb9000a972"
server_key_version: "encryptionkey@0"
I20260809 16:20:40.960321 28199 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786292417722232-21710-0/minicluster-data/ts-0/wal/instance:
uuid: "896c9bb9bb2a48599f9e66bda347b48c"
format_stamp: "Formatted at 2026-08-09 16:20:40 on dist-test-slave-s11t"
server_key: "41a24985b70f8e6b12f5c40c37a5491e"
server_key_iv: "618dab347a7e2d034b5f48fb9000a972"
server_key_version: "encryptionkey@0"
I20260809 16:20:40.961774 28199 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 16:20:40.962589 28226 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:20:40.962777 28199 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 16:20:40.962906 28199 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786292417722232-21710-0/minicluster-data/ts-0/data,/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786292417722232-21710-0/minicluster-data/ts-0/wal
uuid: "896c9bb9bb2a48599f9e66bda347b48c"
format_stamp: "Formatted at 2026-08-09 16:20:40 on dist-test-slave-s11t"
server_key: "41a24985b70f8e6b12f5c40c37a5491e"
server_key_iv: "618dab347a7e2d034b5f48fb9000a972"
server_key_version: "encryptionkey@0"
I20260809 16:20:40.962996 28199 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786292417722232-21710-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786292417722232-21710-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786292417722232-21710-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:20:40.984576 28199 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:20:40.987162 28199 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:20:40.987344 28199 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:20:40.987610 28199 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:20:40.987957 28199 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:20:40.988013 28199 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:20:40.988046 28199 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:20:40.988075 28199 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:20:40.993907 28199 rpc_server.cc:307] RPC server started. Bound to: 127.21.51.129:42505
I20260809 16:20:40.993945 28405 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.51.129:42505 every 8 connection(s)
I20260809 16:20:40.994238 28199 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786292417722232-21710-0/minicluster-data/ts-0/data/info.pb
I20260809 16:20:40.997079 21710 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrNqUCn/build/release/bin/kudu as pid 28199
I20260809 16:20:40.997211 21710 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786292417722232-21710-0/minicluster-data/ts-0/wal/instance
I20260809 16:20:40.997399 21710 external_mini_cluster.cc:1428] Setting key 6b8863af9d25a44138dfee261d8f6334
Aug 09 16:20:40 dist-test-slave-s11t krb5kdc[28030](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292440, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.51.129@KRBTEST.COM for kudu/127.21.51.190@KRBTEST.COM
I20260809 16:20:41.005481 28158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:40.994951 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.51.129:35391 (local address 127.21.51.190:42777)
0809 16:20:40.995012 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0809 16:20:40.995014 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:20:40.995239 (+ 225us) server_negotiation.cc:407] Connection header received
0809 16:20:40.995769 (+ 530us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:20:40.995770 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:20:40.995783 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:20:40.995841 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:20:40.997218 (+ 1377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:40.997906 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:40.998975 (+ 1069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:40.999315 (+ 340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:40.999392 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:41.002008 (+ 2616us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:20:41.002009 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:20:41.002010 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:20:41.002017 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:20:41.003640 (+ 1623us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:20:41.004253 (+ 613us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:20:41.004255 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:20:41.004255 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:20:41.004328 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:20:41.004640 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:20:41.004641 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:20:41.004642 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:20:41.004869 (+ 227us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:20:41.004976 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0809 16:20:41.005276 (+ 300us) server_negotiation.cc:299] Negotiation successful
0809 16:20:41.005372 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260809 16:20:41.005679 28409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:40.994908 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:42777 (local address 127.21.51.129:35391)
0809 16:20:40.995104 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0809 16:20:40.995166 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0809 16:20:40.995658 (+ 492us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:20:40.995910 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:20:40.995914 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:20:40.996328 (+ 414us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:20:40.997130 (+ 802us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:20:40.997137 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:20:40.998046 (+ 909us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:20:40.998047 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:20:40.998882 (+ 835us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:20:40.998884 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:20:40.999406 (+ 522us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:20:40.999407 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:20:40.999513 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:41.000096 (+ 583us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:20:41.000104 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:20:41.001890 (+ 1786us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:20:41.003765 (+ 1875us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:20:41.003773 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:20:41.003775 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:20:41.004118 (+ 343us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:20:41.004443 (+ 325us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:20:41.004444 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:20:41.004445 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:20:41.004566 (+ 121us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:20:41.004951 (+ 385us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:20:41.004958 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:20:41.005115 (+ 157us) client_negotiation.cc:769] Sending connection context
0809 16:20:41.005233 (+ 118us) client_negotiation.cc:240] Negotiation successful
0809 16:20:41.005429 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":97,"threads_started":1}
I20260809 16:20:41.006263 28406 heartbeater.cc:347] Connected to a master server at 127.21.51.190:42777
I20260809 16:20:41.006381 28406 heartbeater.cc:464] Registering TS with master...
I20260809 16:20:41.006556 28406 heartbeater.cc:511] Master 127.21.51.190:42777 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.21.51.130@KRBTEST.COM; defaulting to no policy
I20260809 16:20:41.007102 28102 ts_manager.cc:194] Registered new tserver with Master: 896c9bb9bb2a48599f9e66bda347b48c (127.21.51.129:42505)
I20260809 16:20:41.007740 28102 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.51.129@KRBTEST.COM'} at 127.21.51.129:35391
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.51.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.51.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.130.keytab.
Entry for principal kudu/127.21.51.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.130.keytab.
I20260809 16:20:41.025147 21710 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrNqUCn/build/release/bin/kudu
/tmp/dist-test-taskrNqUCn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786292417722232-21710-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786292417722232-21710-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786292417722232-21710-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786292417722232-21710-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.51.130:0
--local_ip_for_outbound_sockets=127.21.51.130
--webserver_interface=127.21.51.130
--webserver_port=0
--tserver_master_addrs=127.21.51.190:42777
--keytab_file=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.130.keytab
--principal=kudu/127.21.51.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.51.148:43673
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786292417722232-21710-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786292417722232-21710-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786292417722232-21710-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:20:41.109524 28412 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:20:41.109764 28412 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:20:41.109805 28412 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:20:41.110898 28412 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:20:41.110934 28412 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:20:41.110960 28412 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.51.130
I20260809 16:20:41.112388 28412 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.51.148:43673
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786292417722232-21710-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786292417722232-21710-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.130.keytab
--principal=kudu/127.21.51.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.51.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786292417722232-21710-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.51.130
--webserver_port=0
--tserver_master_addrs=127.21.51.190:42777
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28412
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.51.130
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786292417722232-21710-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:20:41.112563 28412 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:20:41.112802 28412 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:20:41.117601 28420 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:20:41.117614 28417 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:20:41.117604 28427 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:20:41.118018 28412 server_base.cc:1034] running on GCE node
I20260809 16:20:41.118211 28412 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:20:41.118392 28412 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:20:41.119531 28412 hybrid_clock.cc:648] HybridClock initialized: now 1786292441119515 us; error 33 us; skew 500 ppm
Aug 09 16:20:41 dist-test-slave-s11t krb5kdc[28030](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292441, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.51.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:20:41.122061 28412 init.cc:377] Logged in from keytab as kudu/127.21.51.130@KRBTEST.COM (short username kudu)
I20260809 16:20:41.122431 28412 fs_manager.cc:359] Metadata directory not provided
I20260809 16:20:41.122483 28412 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:20:41.122651 28412 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:20:41.123660 28412 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786292417722232-21710-0/minicluster-data/ts-1/data/instance:
uuid: "9594d8529895463ba8fec97ec85f2ed2"
format_stamp: "Formatted at 2026-08-09 16:20:41 on dist-test-slave-s11t"
server_key: "b6ee00eb738eff5bdf17156c061fb649"
server_key_iv: "ccd7f92f9aa46178c70fbad6562cc3e9"
server_key_version: "encryptionkey@0"
I20260809 16:20:41.124023 28412 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786292417722232-21710-0/minicluster-data/ts-1/wal/instance:
uuid: "9594d8529895463ba8fec97ec85f2ed2"
format_stamp: "Formatted at 2026-08-09 16:20:41 on dist-test-slave-s11t"
server_key: "b6ee00eb738eff5bdf17156c061fb649"
server_key_iv: "ccd7f92f9aa46178c70fbad6562cc3e9"
server_key_version: "encryptionkey@0"
I20260809 16:20:41.125355 28412 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 16:20:41.126120 28436 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:20:41.126298 28412 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 16:20:41.126405 28412 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786292417722232-21710-0/minicluster-data/ts-1/data,/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786292417722232-21710-0/minicluster-data/ts-1/wal
uuid: "9594d8529895463ba8fec97ec85f2ed2"
format_stamp: "Formatted at 2026-08-09 16:20:41 on dist-test-slave-s11t"
server_key: "b6ee00eb738eff5bdf17156c061fb649"
server_key_iv: "ccd7f92f9aa46178c70fbad6562cc3e9"
server_key_version: "encryptionkey@0"
I20260809 16:20:41.126477 28412 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786292417722232-21710-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786292417722232-21710-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786292417722232-21710-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:20:41.133744 28412 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:20:41.135438 28412 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:20:41.135600 28412 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:20:41.135818 28412 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:20:41.136112 28412 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:20:41.136166 28412 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:20:41.136206 28412 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:20:41.136236 28412 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:20:41.142174 28412 rpc_server.cc:307] RPC server started. Bound to: 127.21.51.130:44341
I20260809 16:20:41.142220 28622 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.51.130:44341 every 8 connection(s)
I20260809 16:20:41.142464 28412 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786292417722232-21710-0/minicluster-data/ts-1/data/info.pb
Aug 09 16:20:41 dist-test-slave-s11t krb5kdc[28030](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292441, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.51.130@KRBTEST.COM for kudu/127.21.51.190@KRBTEST.COM
I20260809 16:20:41.150436 21710 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrNqUCn/build/release/bin/kudu as pid 28412
I20260809 16:20:41.150549 21710 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786292417722232-21710-0/minicluster-data/ts-1/wal/instance
I20260809 16:20:41.150744 21710 external_mini_cluster.cc:1428] Setting key 9cc42ac159a4d571f53d3f462c359c63
I20260809 16:20:41.152019 28628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:41.143127 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:42777 (local address 127.21.51.130:58763)
0809 16:20:41.143285 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0809 16:20:41.143325 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0809 16:20:41.143761 (+ 436us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:20:41.143956 (+ 195us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:20:41.143959 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:20:41.144205 (+ 246us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:20:41.144707 (+ 502us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:20:41.144712 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:20:41.145558 (+ 846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:20:41.145559 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:20:41.146356 (+ 797us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:20:41.146358 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:20:41.146749 (+ 391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:20:41.146750 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:20:41.146848 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:41.147325 (+ 477us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:20:41.147333 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:20:41.148671 (+ 1338us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:20:41.150529 (+ 1858us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:20:41.150537 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:20:41.150539 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:20:41.150833 (+ 294us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:20:41.151062 (+ 229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:20:41.151063 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:20:41.151064 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:20:41.151192 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:20:41.151514 (+ 322us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:20:41.151520 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:20:41.151636 (+ 116us) client_negotiation.cc:769] Sending connection context
0809 16:20:41.151721 (+ 85us) client_negotiation.cc:240] Negotiation successful
0809 16:20:41.151841 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":90,"threads_started":1}
I20260809 16:20:41.152015 28158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:41.143170 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.51.130:58763 (local address 127.21.51.190:42777)
0809 16:20:41.143242 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0809 16:20:41.143244 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:20:41.143415 (+ 171us) server_negotiation.cc:407] Connection header received
0809 16:20:41.143840 (+ 425us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:20:41.143841 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:20:41.143851 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:20:41.143907 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:20:41.144853 (+ 946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:41.145465 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:41.146467 (+ 1002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:41.146660 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:41.146732 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:41.148839 (+ 2107us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:20:41.148840 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:20:41.148840 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:20:41.148847 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:20:41.150427 (+ 1580us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:20:41.150928 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:20:41.150929 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:20:41.150929 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:20:41.150982 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:20:41.151285 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:20:41.151286 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:20:41.151286 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:20:41.151406 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:20:41.151499 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0809 16:20:41.151776 (+ 277us) server_negotiation.cc:299] Negotiation successful
0809 16:20:41.151883 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260809 16:20:41.152837 28625 heartbeater.cc:347] Connected to a master server at 127.21.51.190:42777
I20260809 16:20:41.152947 28625 heartbeater.cc:464] Registering TS with master...
I20260809 16:20:41.153111 28625 heartbeater.cc:511] Master 127.21.51.190:42777 requested a full tablet report, sending...
I20260809 16:20:41.153563 28102 ts_manager.cc:194] Registered new tserver with Master: 9594d8529895463ba8fec97ec85f2ed2 (127.21.51.130:44341)
I20260809 16:20:41.154101 28102 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.51.130@KRBTEST.COM'} at 127.21.51.130:58763
WARNING: no policy specified for kudu/127.21.51.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.51.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.51.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.131.keytab.
Entry for principal kudu/127.21.51.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.131.keytab.
I20260809 16:20:41.175765 21710 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrNqUCn/build/release/bin/kudu
/tmp/dist-test-taskrNqUCn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786292417722232-21710-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786292417722232-21710-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786292417722232-21710-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786292417722232-21710-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.51.131:0
--local_ip_for_outbound_sockets=127.21.51.131
--webserver_interface=127.21.51.131
--webserver_port=0
--tserver_master_addrs=127.21.51.190:42777
--keytab_file=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.131.keytab
--principal=kudu/127.21.51.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.51.148:43673
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786292417722232-21710-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786292417722232-21710-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786292417722232-21710-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:20:41.260221 28632 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:20:41.260416 28632 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:20:41.260439 28632 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:20:41.261531 28632 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:20:41.261569 28632 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:20:41.261595 28632 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.51.131
I20260809 16:20:41.263044 28632 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.51.148:43673
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786292417722232-21710-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786292417722232-21710-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.131.keytab
--principal=kudu/127.21.51.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.51.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786292417722232-21710-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.51.131
--webserver_port=0
--tserver_master_addrs=127.21.51.190:42777
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28632
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.51.131
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786292417722232-21710-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:20:41.263237 28632 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:20:41.263404 28632 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:20:41.268576 28642 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:20:41.268695 28632 server_base.cc:1034] running on GCE node
W20260809 16:20:41.268652 28644 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:20:41.268564 28641 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:20:41.269155 28632 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:20:41.269433 28632 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:20:41.270622 28632 hybrid_clock.cc:648] HybridClock initialized: now 1786292441270605 us; error 28 us; skew 500 ppm
Aug 09 16:20:41 dist-test-slave-s11t krb5kdc[28030](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292441, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.51.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:20:41.273322 28632 init.cc:377] Logged in from keytab as kudu/127.21.51.131@KRBTEST.COM (short username kudu)
I20260809 16:20:41.273701 28632 fs_manager.cc:359] Metadata directory not provided
I20260809 16:20:41.273820 28632 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:20:41.273977 28632 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:20:41.275002 28632 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786292417722232-21710-0/minicluster-data/ts-2/data/instance:
uuid: "657832c2f61d4116a413a51cafcc0354"
format_stamp: "Formatted at 2026-08-09 16:20:41 on dist-test-slave-s11t"
server_key: "eac66f89fc05c8251ae4d85cb4a09ff0"
server_key_iv: "eee870d5493b2fc4c14e6de927d9d0e4"
server_key_version: "encryptionkey@0"
I20260809 16:20:41.275367 28632 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786292417722232-21710-0/minicluster-data/ts-2/wal/instance:
uuid: "657832c2f61d4116a413a51cafcc0354"
format_stamp: "Formatted at 2026-08-09 16:20:41 on dist-test-slave-s11t"
server_key: "eac66f89fc05c8251ae4d85cb4a09ff0"
server_key_iv: "eee870d5493b2fc4c14e6de927d9d0e4"
server_key_version: "encryptionkey@0"
I20260809 16:20:41.277036 28632 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260809 16:20:41.277794 28651 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:20:41.278040 28632 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:20:41.278141 28632 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786292417722232-21710-0/minicluster-data/ts-2/data,/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786292417722232-21710-0/minicluster-data/ts-2/wal
uuid: "657832c2f61d4116a413a51cafcc0354"
format_stamp: "Formatted at 2026-08-09 16:20:41 on dist-test-slave-s11t"
server_key: "eac66f89fc05c8251ae4d85cb4a09ff0"
server_key_iv: "eee870d5493b2fc4c14e6de927d9d0e4"
server_key_version: "encryptionkey@0"
I20260809 16:20:41.278229 28632 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786292417722232-21710-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786292417722232-21710-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786292417722232-21710-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:20:41.292219 28632 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:20:41.294215 28632 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:20:41.294390 28632 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:20:41.294612 28632 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:20:41.294920 28632 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:20:41.294958 28632 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:20:41.294981 28632 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:20:41.294994 28632 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:20:41.300843 28632 rpc_server.cc:307] RPC server started. Bound to: 127.21.51.131:45617
I20260809 16:20:41.300917 28822 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.51.131:45617 every 8 connection(s)
I20260809 16:20:41.301179 28632 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786292417722232-21710-0/minicluster-data/ts-2/data/info.pb
I20260809 16:20:41.301338 21710 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrNqUCn/build/release/bin/kudu as pid 28632
I20260809 16:20:41.301460 21710 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786292417722232-21710-0/minicluster-data/ts-2/wal/instance
I20260809 16:20:41.301683 21710 external_mini_cluster.cc:1428] Setting key c0ec45a3d62fe20f30cef2769e8ab5da
Aug 09 16:20:41 dist-test-slave-s11t krb5kdc[28030](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292441, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.51.131@KRBTEST.COM for kudu/127.21.51.190@KRBTEST.COM
I20260809 16:20:41.310811 28158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:41.301901 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.51.131:50005 (local address 127.21.51.190:42777)
0809 16:20:41.301952 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0809 16:20:41.301954 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:20:41.302326 (+ 372us) server_negotiation.cc:407] Connection header received
0809 16:20:41.302896 (+ 570us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:20:41.302897 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:20:41.302907 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:20:41.302956 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:20:41.303830 (+ 874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:41.304530 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:41.305785 (+ 1255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:41.306019 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:41.306126 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:41.308156 (+ 2030us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:20:41.308157 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:20:41.308157 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:20:41.308164 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:20:41.309685 (+ 1521us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:20:41.310059 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:20:41.310060 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:20:41.310061 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:20:41.310114 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:20:41.310327 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:20:41.310328 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:20:41.310328 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:20:41.310427 (+ 99us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:20:41.310478 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0809 16:20:41.310680 (+ 202us) server_negotiation.cc:299] Negotiation successful
0809 16:20:41.310748 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260809 16:20:41.311071 28828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:41.301894 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:42777 (local address 127.21.51.131:50005)
0809 16:20:41.302129 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0809 16:20:41.302169 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0809 16:20:41.302824 (+ 655us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:20:41.303004 (+ 180us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:20:41.303007 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:20:41.303255 (+ 248us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:20:41.303731 (+ 476us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:20:41.303737 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:20:41.304615 (+ 878us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:20:41.304616 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:20:41.305706 (+ 1090us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:20:41.305709 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:20:41.306141 (+ 432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:20:41.306142 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:20:41.306264 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:41.306671 (+ 407us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:20:41.306679 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:20:41.308056 (+ 1377us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:20:41.309769 (+ 1713us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:20:41.309775 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:20:41.309777 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:20:41.309982 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:20:41.310175 (+ 193us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:20:41.310176 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:20:41.310176 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:20:41.310271 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:20:41.310482 (+ 211us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:20:41.310486 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:20:41.310589 (+ 103us) client_negotiation.cc:769] Sending connection context
0809 16:20:41.310667 (+ 78us) client_negotiation.cc:240] Negotiation successful
0809 16:20:41.310813 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":110,"threads_started":1}
I20260809 16:20:41.311574 28826 heartbeater.cc:347] Connected to a master server at 127.21.51.190:42777
I20260809 16:20:41.311681 28826 heartbeater.cc:464] Registering TS with master...
I20260809 16:20:41.311869 28826 heartbeater.cc:511] Master 127.21.51.190:42777 requested a full tablet report, sending...
I20260809 16:20:41.312230 28102 ts_manager.cc:194] Registered new tserver with Master: 657832c2f61d4116a413a51cafcc0354 (127.21.51.131:45617)
I20260809 16:20:41.312606 28102 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.51.131@KRBTEST.COM'} at 127.21.51.131:50005
I20260809 16:20:41.313545 21710 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 16:20:41.320173 28158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:41.314728 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41166 (local address 127.21.51.190:42777)
0809 16:20:41.314784 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0809 16:20:41.314786 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:20:41.314879 (+ 93us) server_negotiation.cc:407] Connection header received
0809 16:20:41.314928 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:20:41.314929 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:20:41.314936 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:20:41.314983 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:20:41.315479 (+ 496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:41.315940 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:41.316730 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:41.316909 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:41.316961 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:41.318004 (+ 1043us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:20:41.318005 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:20:41.318005 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:20:41.318011 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:20:41.319335 (+ 1324us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:20:41.319626 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:20:41.319626 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:20:41.319627 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:20:41.319665 (+ 38us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:20:41.319792 (+ 127us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:20:41.319792 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:20:41.319792 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:20:41.319910 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:20:41.319981 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0809 16:20:41.320068 (+ 87us) server_negotiation.cc:299] Negotiation successful
0809 16:20:41.320120 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260809 16:20:41.321777 28102 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41166:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 16:20:41.323321 28102 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 16:20:41.330000 28840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:41.325540 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.130:44341 (local address 127.0.0.1:40444)
0809 16:20:41.326192 (+ 652us) negotiation.cc:107] Waiting for socket to connect
0809 16:20:41.326201 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 16:20:41.326246 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:20:41.326549 (+ 303us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:20:41.326550 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:20:41.326552 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:20:41.326602 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:20:41.326604 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:20:41.327488 (+ 884us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:20:41.327490 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:20:41.328990 (+ 1500us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:20:41.328992 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:20:41.329695 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:20:41.329696 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:20:41.329775 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:41.329790 (+ 15us) client_negotiation.cc:769] Sending connection context
0809 16:20:41.329859 (+ 69us) client_negotiation.cc:240] Negotiation successful
0809 16:20:41.329914 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":641,"mutex_wait_us":361,"thread_start_us":56,"threads_started":1}
I20260809 16:20:41.330147 28849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:41.325570 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.129:42505 (local address 127.0.0.1:35874)
0809 16:20:41.326061 (+ 491us) negotiation.cc:107] Waiting for socket to connect
0809 16:20:41.326090 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0809 16:20:41.326156 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:20:41.326572 (+ 416us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:20:41.326573 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:20:41.326575 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:20:41.326605 (+ 30us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:20:41.326606 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:20:41.327832 (+ 1226us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:20:41.327834 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:20:41.328878 (+ 1044us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:20:41.328882 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:20:41.329832 (+ 950us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:20:41.329833 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:20:41.329890 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:41.329894 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 16:20:41.329955 (+ 61us) client_negotiation.cc:240] Negotiation successful
0809 16:20:41.329986 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":480,"thread_start_us":65,"threads_started":1}
I20260809 16:20:41.330117 28843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:41.325705 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35874 (local address 127.21.51.129:42505)
0809 16:20:41.325985 (+ 280us) server_negotiation.cc:206] Beginning negotiation
0809 16:20:41.325990 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 16:20:41.326349 (+ 359us) server_negotiation.cc:407] Connection header received
0809 16:20:41.326398 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:20:41.326400 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:20:41.326415 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:20:41.326488 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:20:41.326696 (+ 208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:41.327736 (+ 1040us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:41.328964 (+ 1228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:41.329750 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:41.329827 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:41.329894 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0809 16:20:41.329965 (+ 71us) server_negotiation.cc:299] Negotiation successful
0809 16:20:41.330008 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":244,"thread_start_us":95,"threads_started":1}
I20260809 16:20:41.330551 28841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:41.325662 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57392 (local address 127.21.51.131:45617)
0809 16:20:41.325811 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0809 16:20:41.325815 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 16:20:41.326110 (+ 295us) server_negotiation.cc:407] Connection header received
0809 16:20:41.326154 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:20:41.326156 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:20:41.326172 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:20:41.326249 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:20:41.326428 (+ 179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:41.327488 (+ 1060us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:41.329029 (+ 1541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:41.329747 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:41.329832 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:41.329882 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0809 16:20:41.330408 (+ 526us) server_negotiation.cc:299] Negotiation successful
0809 16:20:41.330457 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":120,"thread_start_us":88,"threads_started":1}
I20260809 16:20:41.330710 28846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:41.325754 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40444 (local address 127.21.51.130:44341)
0809 16:20:41.325936 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0809 16:20:41.325941 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 16:20:41.326305 (+ 364us) server_negotiation.cc:407] Connection header received
0809 16:20:41.326352 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:20:41.326355 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:20:41.326371 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:20:41.326458 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:20:41.326705 (+ 247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:41.327371 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:41.329077 (+ 1706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:41.329606 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:41.329695 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:41.329757 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0809 16:20:41.330536 (+ 779us) server_negotiation.cc:299] Negotiation successful
0809 16:20:41.330592 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":117,"thread_start_us":82,"threads_started":1}
I20260809 16:20:41.330747 28839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:41.325540 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.131:45617 (local address 127.0.0.1:57392)
0809 16:20:41.325981 (+ 441us) negotiation.cc:107] Waiting for socket to connect
0809 16:20:41.326020 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0809 16:20:41.326095 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:20:41.326249 (+ 154us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:20:41.326252 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:20:41.326254 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:20:41.326321 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:20:41.326323 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:20:41.327627 (+ 1304us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:20:41.327628 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:20:41.328953 (+ 1325us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:20:41.328956 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:20:41.329916 (+ 960us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:20:41.329917 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:20:41.329994 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:41.329998 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 16:20:41.330634 (+ 636us) client_negotiation.cc:240] Negotiation successful
0809 16:20:41.330684 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":418,"mutex_wait_us":58,"thread_start_us":59,"threads_started":1}
I20260809 16:20:41.330983 28305 tablet_service.cc:1467] Processing CreateTablet for tablet bb352d6d02ef454797f3cb31d5cf08aa (DEFAULT_TABLE table=test-table [id=e4ac20304a454808abee5c485b648c9a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:20:41.331352 28305 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bb352d6d02ef454797f3cb31d5cf08aa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:20:41.331368 28519 tablet_service.cc:1467] Processing CreateTablet for tablet bb352d6d02ef454797f3cb31d5cf08aa (DEFAULT_TABLE table=test-table [id=e4ac20304a454808abee5c485b648c9a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:20:41.331709 28519 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bb352d6d02ef454797f3cb31d5cf08aa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:20:41.332180 28720 tablet_service.cc:1467] Processing CreateTablet for tablet bb352d6d02ef454797f3cb31d5cf08aa (DEFAULT_TABLE table=test-table [id=e4ac20304a454808abee5c485b648c9a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:20:41.332492 28720 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bb352d6d02ef454797f3cb31d5cf08aa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:20:41.335290 28850 tablet_bootstrap.cc:492] T bb352d6d02ef454797f3cb31d5cf08aa P 896c9bb9bb2a48599f9e66bda347b48c: Bootstrap starting.
I20260809 16:20:41.335637 28851 tablet_bootstrap.cc:492] T bb352d6d02ef454797f3cb31d5cf08aa P 9594d8529895463ba8fec97ec85f2ed2: Bootstrap starting.
I20260809 16:20:41.335778 28852 tablet_bootstrap.cc:492] T bb352d6d02ef454797f3cb31d5cf08aa P 657832c2f61d4116a413a51cafcc0354: Bootstrap starting.
I20260809 16:20:41.336079 28850 tablet_bootstrap.cc:654] T bb352d6d02ef454797f3cb31d5cf08aa P 896c9bb9bb2a48599f9e66bda347b48c: Neither blocks nor log segments found. Creating new log.
I20260809 16:20:41.336406 28852 tablet_bootstrap.cc:654] T bb352d6d02ef454797f3cb31d5cf08aa P 657832c2f61d4116a413a51cafcc0354: Neither blocks nor log segments found. Creating new log.
I20260809 16:20:41.336405 28851 tablet_bootstrap.cc:654] T bb352d6d02ef454797f3cb31d5cf08aa P 9594d8529895463ba8fec97ec85f2ed2: Neither blocks nor log segments found. Creating new log.
I20260809 16:20:41.336774 28850 log.cc:824] T bb352d6d02ef454797f3cb31d5cf08aa P 896c9bb9bb2a48599f9e66bda347b48c: Log is configured to *not* fsync() on all Append() calls
I20260809 16:20:41.337070 28852 log.cc:824] T bb352d6d02ef454797f3cb31d5cf08aa P 657832c2f61d4116a413a51cafcc0354: Log is configured to *not* fsync() on all Append() calls
I20260809 16:20:41.337087 28851 log.cc:824] T bb352d6d02ef454797f3cb31d5cf08aa P 9594d8529895463ba8fec97ec85f2ed2: Log is configured to *not* fsync() on all Append() calls
I20260809 16:20:41.337698 28850 tablet_bootstrap.cc:492] T bb352d6d02ef454797f3cb31d5cf08aa P 896c9bb9bb2a48599f9e66bda347b48c: No bootstrap required, opened a new log
I20260809 16:20:41.337783 28850 ts_tablet_manager.cc:1397] T bb352d6d02ef454797f3cb31d5cf08aa P 896c9bb9bb2a48599f9e66bda347b48c: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20260809 16:20:41.337801 28851 tablet_bootstrap.cc:492] T bb352d6d02ef454797f3cb31d5cf08aa P 9594d8529895463ba8fec97ec85f2ed2: No bootstrap required, opened a new log
I20260809 16:20:41.337864 28851 ts_tablet_manager.cc:1397] T bb352d6d02ef454797f3cb31d5cf08aa P 9594d8529895463ba8fec97ec85f2ed2: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 16:20:41.338068 28852 tablet_bootstrap.cc:492] T bb352d6d02ef454797f3cb31d5cf08aa P 657832c2f61d4116a413a51cafcc0354: No bootstrap required, opened a new log
I20260809 16:20:41.338141 28852 ts_tablet_manager.cc:1397] T bb352d6d02ef454797f3cb31d5cf08aa P 657832c2f61d4116a413a51cafcc0354: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 16:20:41.339303 28851 raft_consensus.cc:348] T bb352d6d02ef454797f3cb31d5cf08aa P 9594d8529895463ba8fec97ec85f2ed2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9594d8529895463ba8fec97ec85f2ed2" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 44341 } } peers { permanent_uuid: "657832c2f61d4116a413a51cafcc0354" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 45617 } } peers { permanent_uuid: "896c9bb9bb2a48599f9e66bda347b48c" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 42505 } }
I20260809 16:20:41.339466 28851 raft_consensus.cc:374] T bb352d6d02ef454797f3cb31d5cf08aa P 9594d8529895463ba8fec97ec85f2ed2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:20:41.339514 28851 raft_consensus.cc:729] T bb352d6d02ef454797f3cb31d5cf08aa P 9594d8529895463ba8fec97ec85f2ed2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9594d8529895463ba8fec97ec85f2ed2, State: Initialized, Role: FOLLOWER
I20260809 16:20:41.339664 28851 consensus_queue.cc:260] T bb352d6d02ef454797f3cb31d5cf08aa P 9594d8529895463ba8fec97ec85f2ed2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9594d8529895463ba8fec97ec85f2ed2" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 44341 } } peers { permanent_uuid: "657832c2f61d4116a413a51cafcc0354" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 45617 } } peers { permanent_uuid: "896c9bb9bb2a48599f9e66bda347b48c" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 42505 } }
I20260809 16:20:41.339814 28850 raft_consensus.cc:348] T bb352d6d02ef454797f3cb31d5cf08aa P 896c9bb9bb2a48599f9e66bda347b48c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9594d8529895463ba8fec97ec85f2ed2" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 44341 } } peers { permanent_uuid: "657832c2f61d4116a413a51cafcc0354" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 45617 } } peers { permanent_uuid: "896c9bb9bb2a48599f9e66bda347b48c" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 42505 } }
I20260809 16:20:41.339944 28851 ts_tablet_manager.cc:1428] T bb352d6d02ef454797f3cb31d5cf08aa P 9594d8529895463ba8fec97ec85f2ed2: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260809 16:20:41.339991 28850 raft_consensus.cc:374] T bb352d6d02ef454797f3cb31d5cf08aa P 896c9bb9bb2a48599f9e66bda347b48c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:20:41.340034 28850 raft_consensus.cc:729] T bb352d6d02ef454797f3cb31d5cf08aa P 896c9bb9bb2a48599f9e66bda347b48c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 896c9bb9bb2a48599f9e66bda347b48c, State: Initialized, Role: FOLLOWER
I20260809 16:20:41.340083 28625 heartbeater.cc:503] Master 127.21.51.190:42777 was elected leader, sending a full tablet report...
I20260809 16:20:41.340049 28852 raft_consensus.cc:348] T bb352d6d02ef454797f3cb31d5cf08aa P 657832c2f61d4116a413a51cafcc0354 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9594d8529895463ba8fec97ec85f2ed2" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 44341 } } peers { permanent_uuid: "657832c2f61d4116a413a51cafcc0354" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 45617 } } peers { permanent_uuid: "896c9bb9bb2a48599f9e66bda347b48c" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 42505 } }
I20260809 16:20:41.340153 28850 consensus_queue.cc:260] T bb352d6d02ef454797f3cb31d5cf08aa P 896c9bb9bb2a48599f9e66bda347b48c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9594d8529895463ba8fec97ec85f2ed2" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 44341 } } peers { permanent_uuid: "657832c2f61d4116a413a51cafcc0354" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 45617 } } peers { permanent_uuid: "896c9bb9bb2a48599f9e66bda347b48c" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 42505 } }
I20260809 16:20:41.340193 28852 raft_consensus.cc:374] T bb352d6d02ef454797f3cb31d5cf08aa P 657832c2f61d4116a413a51cafcc0354 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:20:41.340229 28852 raft_consensus.cc:729] T bb352d6d02ef454797f3cb31d5cf08aa P 657832c2f61d4116a413a51cafcc0354 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 657832c2f61d4116a413a51cafcc0354, State: Initialized, Role: FOLLOWER
I20260809 16:20:41.340342 28852 consensus_queue.cc:260] T bb352d6d02ef454797f3cb31d5cf08aa P 657832c2f61d4116a413a51cafcc0354 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9594d8529895463ba8fec97ec85f2ed2" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 44341 } } peers { permanent_uuid: "657832c2f61d4116a413a51cafcc0354" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 45617 } } peers { permanent_uuid: "896c9bb9bb2a48599f9e66bda347b48c" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 42505 } }
I20260809 16:20:41.340425 28850 ts_tablet_manager.cc:1428] T bb352d6d02ef454797f3cb31d5cf08aa P 896c9bb9bb2a48599f9e66bda347b48c: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260809 16:20:41.340543 28406 heartbeater.cc:503] Master 127.21.51.190:42777 was elected leader, sending a full tablet report...
I20260809 16:20:41.340634 28852 ts_tablet_manager.cc:1428] T bb352d6d02ef454797f3cb31d5cf08aa P 657832c2f61d4116a413a51cafcc0354: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260809 16:20:41.340776 28826 heartbeater.cc:503] Master 127.21.51.190:42777 was elected leader, sending a full tablet report...
W20260809 16:20:41.393059 28626 tablet.cc:2369] T bb352d6d02ef454797f3cb31d5cf08aa P 9594d8529895463ba8fec97ec85f2ed2: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 16:20:41.486838 28857 raft_consensus.cc:482] T bb352d6d02ef454797f3cb31d5cf08aa P 896c9bb9bb2a48599f9e66bda347b48c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 16:20:41.486987 28857 raft_consensus.cc:504] T bb352d6d02ef454797f3cb31d5cf08aa P 896c9bb9bb2a48599f9e66bda347b48c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9594d8529895463ba8fec97ec85f2ed2" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 44341 } } peers { permanent_uuid: "657832c2f61d4116a413a51cafcc0354" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 45617 } } peers { permanent_uuid: "896c9bb9bb2a48599f9e66bda347b48c" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 42505 } }
I20260809 16:20:41.487298 28857 leader_election.cc:288] T bb352d6d02ef454797f3cb31d5cf08aa P 896c9bb9bb2a48599f9e66bda347b48c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9594d8529895463ba8fec97ec85f2ed2 (127.21.51.130:44341), 657832c2f61d4116a413a51cafcc0354 (127.21.51.131:45617)
I20260809 16:20:41.490274 28409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:41.487490 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.131:45617 (local address 127.21.51.129:47655)
0809 16:20:41.487542 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0809 16:20:41.487553 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 16:20:41.487597 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:20:41.487804 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:20:41.487806 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:20:41.487808 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:20:41.487867 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:20:41.487869 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:20:41.488595 (+ 726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:20:41.488596 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:20:41.489418 (+ 822us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:20:41.489421 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:20:41.490037 (+ 616us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:20:41.490038 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:20:41.490109 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:41.490114 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 16:20:41.490170 (+ 56us) client_negotiation.cc:240] Negotiation successful
0809 16:20:41.490209 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33}
I20260809 16:20:41.490319 28841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:41.487605 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.51.129:47655 (local address 127.21.51.131:45617)
0809 16:20:41.487667 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0809 16:20:41.487669 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:20:41.487685 (+ 16us) server_negotiation.cc:407] Connection header received
0809 16:20:41.487710 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:20:41.487711 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:20:41.487718 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:20:41.487797 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:20:41.487963 (+ 166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:41.488516 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:41.489509 (+ 993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:41.489958 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:41.490017 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:41.490025 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0809 16:20:41.490189 (+ 164us) server_negotiation.cc:299] Negotiation successful
0809 16:20:41.490232 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260809 16:20:41.490469 28859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:41.487491 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.130:44341 (local address 127.21.51.129:33723)
0809 16:20:41.487802 (+ 311us) negotiation.cc:107] Waiting for socket to connect
0809 16:20:41.487812 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 16:20:41.487869 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:20:41.487987 (+ 118us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:20:41.487988 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:20:41.487989 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:20:41.488062 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:20:41.488063 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:20:41.488759 (+ 696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:20:41.488760 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:20:41.489568 (+ 808us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:20:41.489570 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:20:41.490194 (+ 624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:20:41.490195 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:20:41.490272 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:41.490276 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 16:20:41.490334 (+ 58us) client_negotiation.cc:240] Negotiation successful
0809 16:20:41.490374 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":298,"mutex_wait_us":157,"thread_start_us":117,"threads_started":1}
I20260809 16:20:41.490490 28846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:41.487627 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.51.129:33723 (local address 127.21.51.130:44341)
0809 16:20:41.487674 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0809 16:20:41.487677 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 16:20:41.487867 (+ 190us) server_negotiation.cc:407] Connection header received
0809 16:20:41.487906 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:20:41.487907 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:20:41.487920 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:20:41.487976 (+ 56us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:20:41.488126 (+ 150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:41.488607 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:41.489662 (+ 1055us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:41.490084 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:41.490161 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:41.490170 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0809 16:20:41.490356 (+ 186us) server_negotiation.cc:299] Negotiation successful
0809 16:20:41.490402 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260809 16:20:41.490693 28755 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bb352d6d02ef454797f3cb31d5cf08aa" candidate_uuid: "896c9bb9bb2a48599f9e66bda347b48c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "657832c2f61d4116a413a51cafcc0354" is_pre_election: true
I20260809 16:20:41.490851 28755 raft_consensus.cc:2454] T bb352d6d02ef454797f3cb31d5cf08aa P 657832c2f61d4116a413a51cafcc0354 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 896c9bb9bb2a48599f9e66bda347b48c in term 0.
I20260809 16:20:41.490860 28550 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bb352d6d02ef454797f3cb31d5cf08aa" candidate_uuid: "896c9bb9bb2a48599f9e66bda347b48c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9594d8529895463ba8fec97ec85f2ed2" is_pre_election: true
I20260809 16:20:41.491012 28550 raft_consensus.cc:2454] T bb352d6d02ef454797f3cb31d5cf08aa P 9594d8529895463ba8fec97ec85f2ed2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 896c9bb9bb2a48599f9e66bda347b48c in term 0.
I20260809 16:20:41.491114 28230 leader_election.cc:302] T bb352d6d02ef454797f3cb31d5cf08aa P 896c9bb9bb2a48599f9e66bda347b48c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 657832c2f61d4116a413a51cafcc0354, 896c9bb9bb2a48599f9e66bda347b48c; no voters:
I20260809 16:20:41.491245 28857 raft_consensus.cc:2781] T bb352d6d02ef454797f3cb31d5cf08aa P 896c9bb9bb2a48599f9e66bda347b48c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 16:20:41.491304 28857 raft_consensus.cc:482] T bb352d6d02ef454797f3cb31d5cf08aa P 896c9bb9bb2a48599f9e66bda347b48c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 16:20:41.491321 28857 raft_consensus.cc:3037] T bb352d6d02ef454797f3cb31d5cf08aa P 896c9bb9bb2a48599f9e66bda347b48c [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:20:41.491989 28857 raft_consensus.cc:504] T bb352d6d02ef454797f3cb31d5cf08aa P 896c9bb9bb2a48599f9e66bda347b48c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9594d8529895463ba8fec97ec85f2ed2" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 44341 } } peers { permanent_uuid: "657832c2f61d4116a413a51cafcc0354" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 45617 } } peers { permanent_uuid: "896c9bb9bb2a48599f9e66bda347b48c" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 42505 } }
I20260809 16:20:41.492126 28857 leader_election.cc:288] T bb352d6d02ef454797f3cb31d5cf08aa P 896c9bb9bb2a48599f9e66bda347b48c [CANDIDATE]: Term 1 election: Requested vote from peers 9594d8529895463ba8fec97ec85f2ed2 (127.21.51.130:44341), 657832c2f61d4116a413a51cafcc0354 (127.21.51.131:45617)
I20260809 16:20:41.492292 28755 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bb352d6d02ef454797f3cb31d5cf08aa" candidate_uuid: "896c9bb9bb2a48599f9e66bda347b48c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "657832c2f61d4116a413a51cafcc0354"
I20260809 16:20:41.492319 28550 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bb352d6d02ef454797f3cb31d5cf08aa" candidate_uuid: "896c9bb9bb2a48599f9e66bda347b48c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9594d8529895463ba8fec97ec85f2ed2"
I20260809 16:20:41.492385 28755 raft_consensus.cc:3037] T bb352d6d02ef454797f3cb31d5cf08aa P 657832c2f61d4116a413a51cafcc0354 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:20:41.492385 28550 raft_consensus.cc:3037] T bb352d6d02ef454797f3cb31d5cf08aa P 9594d8529895463ba8fec97ec85f2ed2 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:20:41.493364 28755 raft_consensus.cc:2454] T bb352d6d02ef454797f3cb31d5cf08aa P 657832c2f61d4116a413a51cafcc0354 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 896c9bb9bb2a48599f9e66bda347b48c in term 1.
I20260809 16:20:41.493366 28550 raft_consensus.cc:2454] T bb352d6d02ef454797f3cb31d5cf08aa P 9594d8529895463ba8fec97ec85f2ed2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 896c9bb9bb2a48599f9e66bda347b48c in term 1.
I20260809 16:20:41.493595 28229 leader_election.cc:302] T bb352d6d02ef454797f3cb31d5cf08aa P 896c9bb9bb2a48599f9e66bda347b48c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 896c9bb9bb2a48599f9e66bda347b48c, 9594d8529895463ba8fec97ec85f2ed2; no voters:
I20260809 16:20:41.493705 28857 raft_consensus.cc:2781] T bb352d6d02ef454797f3cb31d5cf08aa P 896c9bb9bb2a48599f9e66bda347b48c [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:20:41.493903 28857 raft_consensus.cc:686] T bb352d6d02ef454797f3cb31d5cf08aa P 896c9bb9bb2a48599f9e66bda347b48c [term 1 LEADER]: Becoming Leader. State: Replica: 896c9bb9bb2a48599f9e66bda347b48c, State: Running, Role: LEADER
I20260809 16:20:41.494011 28857 consensus_queue.cc:237] T bb352d6d02ef454797f3cb31d5cf08aa P 896c9bb9bb2a48599f9e66bda347b48c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9594d8529895463ba8fec97ec85f2ed2" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 44341 } } peers { permanent_uuid: "657832c2f61d4116a413a51cafcc0354" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 45617 } } peers { permanent_uuid: "896c9bb9bb2a48599f9e66bda347b48c" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 42505 } }
W20260809 16:20:41.494979 28407 tablet.cc:2369] T bb352d6d02ef454797f3cb31d5cf08aa P 896c9bb9bb2a48599f9e66bda347b48c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 16:20:41.494954 28101 catalog_manager.cc:5495] T bb352d6d02ef454797f3cb31d5cf08aa P 896c9bb9bb2a48599f9e66bda347b48c reported cstate change: term changed from 0 to 1, leader changed from <none> to 896c9bb9bb2a48599f9e66bda347b48c (127.21.51.129). New cstate: current_term: 1 leader_uuid: "896c9bb9bb2a48599f9e66bda347b48c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9594d8529895463ba8fec97ec85f2ed2" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 44341 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "657832c2f61d4116a413a51cafcc0354" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 45617 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "896c9bb9bb2a48599f9e66bda347b48c" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 42505 } health_report { overall_health: HEALTHY } } }
I20260809 16:20:41.517699 28843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:41.515163 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35882 (local address 127.21.51.129:42505)
0809 16:20:41.515256 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0809 16:20:41.515258 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:20:41.515268 (+ 10us) server_negotiation.cc:407] Connection header received
0809 16:20:41.515329 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:20:41.515330 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:20:41.515349 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:20:41.515421 (+ 72us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 16:20:41.515574 (+ 153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:41.516047 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:41.516893 (+ 846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:41.517100 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:41.517175 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:41.517342 (+ 167us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 16:20:41.517449 (+ 107us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 16:20:41.517545 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0809 16:20:41.517564 (+ 19us) server_negotiation.cc:299] Negotiation successful
0809 16:20:41.517587 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260809 16:20:41.519429 28550 raft_consensus.cc:1261] T bb352d6d02ef454797f3cb31d5cf08aa P 9594d8529895463ba8fec97ec85f2ed2 [term 1 FOLLOWER]: Refusing update from remote peer 896c9bb9bb2a48599f9e66bda347b48c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 16:20:41.519506 28755 raft_consensus.cc:1261] T bb352d6d02ef454797f3cb31d5cf08aa P 657832c2f61d4116a413a51cafcc0354 [term 1 FOLLOWER]: Refusing update from remote peer 896c9bb9bb2a48599f9e66bda347b48c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 16:20:41.519827 28857 consensus_queue.cc:1035] T bb352d6d02ef454797f3cb31d5cf08aa P 896c9bb9bb2a48599f9e66bda347b48c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9594d8529895463ba8fec97ec85f2ed2" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 44341 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 16:20:41.519979 28862 consensus_queue.cc:1035] T bb352d6d02ef454797f3cb31d5cf08aa P 896c9bb9bb2a48599f9e66bda347b48c [LEADER]: Connected to new peer: Peer: permanent_uuid: "657832c2f61d4116a413a51cafcc0354" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 45617 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 16:20:41.522235 28866 mvcc.cc:205] Tried to move back new op lower bound from 7316653840461049856 to 7316653840360321024. Current Snapshot: MvccSnapshot[applied={T|T < 7316653840461049856}]
I20260809 16:20:41.522533 28867 mvcc.cc:205] Tried to move back new op lower bound from 7316653840461049856 to 7316653840360321024. Current Snapshot: MvccSnapshot[applied={T|T < 7316653840461049856}]
I20260809 16:20:41.524726 28102 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41166:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 16:20:41.524837 28102 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41166:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 16:20:41.525574 28102 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0d8d83ab4e6f4c539e1291d6fd2d09e3: Sending DeleteTablet for 3 replicas of tablet bb352d6d02ef454797f3cb31d5cf08aa
I20260809 16:20:41.525918 28519 tablet_service.cc:1514] Processing DeleteTablet for tablet bb352d6d02ef454797f3cb31d5cf08aa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:20:41 UTC) from {username='kudu', principal='kudu/127.21.51.190@KRBTEST.COM'} at 127.0.0.1:40444
I20260809 16:20:41.525992 28720 tablet_service.cc:1514] Processing DeleteTablet for tablet bb352d6d02ef454797f3cb31d5cf08aa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:20:41 UTC) from {username='kudu', principal='kudu/127.21.51.190@KRBTEST.COM'} at 127.0.0.1:57392
I20260809 16:20:41.526149 28305 tablet_service.cc:1514] Processing DeleteTablet for tablet bb352d6d02ef454797f3cb31d5cf08aa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:20:41 UTC) from {username='kudu', principal='kudu/127.21.51.190@KRBTEST.COM'} at 127.0.0.1:35874
I20260809 16:20:41.526208 28882 tablet_replica.cc:330] T bb352d6d02ef454797f3cb31d5cf08aa P 657832c2f61d4116a413a51cafcc0354: stopping tablet replica
I20260809 16:20:41.526314 28882 raft_consensus.cc:2229] T bb352d6d02ef454797f3cb31d5cf08aa P 657832c2f61d4116a413a51cafcc0354 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 16:20:41.526414 28882 raft_consensus.cc:2258] T bb352d6d02ef454797f3cb31d5cf08aa P 657832c2f61d4116a413a51cafcc0354 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 16:20:41.526413 28885 tablet_replica.cc:330] T bb352d6d02ef454797f3cb31d5cf08aa P 896c9bb9bb2a48599f9e66bda347b48c: stopping tablet replica
I20260809 16:20:41.526610 21710 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrNqUCn/build/release/bin/kudu with pid 28199
I20260809 16:20:41.526633 28885 raft_consensus.cc:2229] T bb352d6d02ef454797f3cb31d5cf08aa P 896c9bb9bb2a48599f9e66bda347b48c [term 1 LEADER]: Raft consensus shutting down.
I20260809 16:20:41.526813 28885 raft_consensus.cc:2258] T bb352d6d02ef454797f3cb31d5cf08aa P 896c9bb9bb2a48599f9e66bda347b48c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 16:20:41.526873 28882 ts_tablet_manager.cc:1905] T bb352d6d02ef454797f3cb31d5cf08aa P 657832c2f61d4116a413a51cafcc0354: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 16:20:41.528560 28882 ts_tablet_manager.cc:1918] T bb352d6d02ef454797f3cb31d5cf08aa P 657832c2f61d4116a413a51cafcc0354: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 16:20:41.528651 28882 log.cc:1196] T bb352d6d02ef454797f3cb31d5cf08aa P 657832c2f61d4116a413a51cafcc0354: Deleting WAL directory at /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786292417722232-21710-0/minicluster-data/ts-2/wal/wals/bb352d6d02ef454797f3cb31d5cf08aa
I20260809 16:20:41.529044 28882 ts_tablet_manager.cc:1939] T bb352d6d02ef454797f3cb31d5cf08aa P 657832c2f61d4116a413a51cafcc0354: Deleting consensus metadata
I20260809 16:20:41.529497 28082 catalog_manager.cc:4841] TS 657832c2f61d4116a413a51cafcc0354 (127.21.51.131:45617): tablet bb352d6d02ef454797f3cb31d5cf08aa (table test-table [id=e4ac20304a454808abee5c485b648c9a]) successfully deleted
I20260809 16:20:41.533051 28880 tablet_replica.cc:330] T bb352d6d02ef454797f3cb31d5cf08aa P 9594d8529895463ba8fec97ec85f2ed2: stopping tablet replica
I20260809 16:20:41.533190 28880 raft_consensus.cc:2229] T bb352d6d02ef454797f3cb31d5cf08aa P 9594d8529895463ba8fec97ec85f2ed2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 16:20:41.533322 28880 raft_consensus.cc:2258] T bb352d6d02ef454797f3cb31d5cf08aa P 9594d8529895463ba8fec97ec85f2ed2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 16:20:41.533932 28880 ts_tablet_manager.cc:1905] T bb352d6d02ef454797f3cb31d5cf08aa P 9594d8529895463ba8fec97ec85f2ed2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 16:20:41.535411 28880 ts_tablet_manager.cc:1918] T bb352d6d02ef454797f3cb31d5cf08aa P 9594d8529895463ba8fec97ec85f2ed2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 16:20:41.535490 28880 log.cc:1196] T bb352d6d02ef454797f3cb31d5cf08aa P 9594d8529895463ba8fec97ec85f2ed2: Deleting WAL directory at /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786292417722232-21710-0/minicluster-data/ts-1/wal/wals/bb352d6d02ef454797f3cb31d5cf08aa
I20260809 16:20:41.535739 28880 ts_tablet_manager.cc:1939] T bb352d6d02ef454797f3cb31d5cf08aa P 9594d8529895463ba8fec97ec85f2ed2: Deleting consensus metadata
I20260809 16:20:41.536155 28081 catalog_manager.cc:4841] TS 9594d8529895463ba8fec97ec85f2ed2 (127.21.51.130:44341): tablet bb352d6d02ef454797f3cb31d5cf08aa (table test-table [id=e4ac20304a454808abee5c485b648c9a]) successfully deleted
W20260809 16:20:41.537825 28084 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.21.51.129:42505 (error 108)
I20260809 16:20:41.538089 21710 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrNqUCn/build/release/bin/kudu with pid 28412
I20260809 16:20:41.541534 28840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:41.538234 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.129:42505 (local address 127.0.0.1:35886)
0809 16:20:41.541422 (+ 3188us) negotiation.cc:107] Waiting for socket to connect
0809 16:20:41.541486 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.51.129:42505: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":3159}
W20260809 16:20:41.544543 28084 catalog_manager.cc:4584] TS 896c9bb9bb2a48599f9e66bda347b48c (127.21.51.129:42505): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet bb352d6d02ef454797f3cb31d5cf08aa: Network error: Client connection negotiation failed: client connection to 127.21.51.129:42505: connect: Connection refused (error 111)
I20260809 16:20:41.549769 21710 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrNqUCn/build/release/bin/kudu with pid 28632
I20260809 16:20:41.560405 21710 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrNqUCn/build/release/bin/kudu with pid 28058
2026-08-09T16:20:41Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (2253 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-08-09T16:20:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T16:20:41Z Disabled control of system clock
I20260809 16:20:41.590162 21710 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrNqUCn/build/release/bin/kudu
/tmp/dist-test-taskrNqUCn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786292417722232-21710-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786292417722232-21710-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786292417722232-21710-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786292417722232-21710-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.51.190:42739
--webserver_interface=127.21.51.190
--webserver_port=0
--builtin_ntp_servers=127.21.51.148:34031
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.51.190:42739
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260809 16:20:41.671928 28890 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:20:41.672118 28890 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:20:41.672139 28890 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:20:41.673216 28890 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 16:20:41.673250 28890 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:20:41.673264 28890 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:20:41.673275 28890 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 16:20:41.673287 28890 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 16:20:41.674571 28890 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.51.148:34031
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786292417722232-21710-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786292417722232-21710-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.51.190:42739
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.51.190:42739
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786292417722232-21710-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.51.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28890
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786292417722232-21710-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:20:41.674763 28890 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:20:41.674997 28890 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:20:41.679919 28900 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:20:41.679930 28902 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:20:41.679958 28890 server_base.cc:1034] running on GCE node
W20260809 16:20:41.679896 28899 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:20:41.680348 28890 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:20:41.680644 28890 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:20:41.681826 28890 hybrid_clock.cc:648] HybridClock initialized: now 1786292441681803 us; error 39 us; skew 500 ppm
I20260809 16:20:41.683121 28890 webserver.cc:460] Webserver started at http://127.21.51.190:35129/ using document root <none> and password file <none>
I20260809 16:20:41.683367 28890 fs_manager.cc:359] Metadata directory not provided
I20260809 16:20:41.683442 28890 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:20:41.683597 28890 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:20:41.684628 28890 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786292417722232-21710-0/minicluster-data/master-0/data/instance:
uuid: "7c2c6ec5f2fa42a7af8c72505053c32d"
format_stamp: "Formatted at 2026-08-09 16:20:41 on dist-test-slave-s11t"
server_key: "d72d698f408ac67682e234b338bd4cb9"
server_key_iv: "a75f6b15d5eb3507c99dbb1caaa3ae9d"
server_key_version: "encryptionkey@0"
I20260809 16:20:41.685045 28890 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786292417722232-21710-0/minicluster-data/master-0/wal/instance:
uuid: "7c2c6ec5f2fa42a7af8c72505053c32d"
format_stamp: "Formatted at 2026-08-09 16:20:41 on dist-test-slave-s11t"
server_key: "d72d698f408ac67682e234b338bd4cb9"
server_key_iv: "a75f6b15d5eb3507c99dbb1caaa3ae9d"
server_key_version: "encryptionkey@0"
I20260809 16:20:41.686354 28890 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:20:41.687057 28912 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:20:41.687232 28890 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 16:20:41.687333 28890 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786292417722232-21710-0/minicluster-data/master-0/data,/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786292417722232-21710-0/minicluster-data/master-0/wal
uuid: "7c2c6ec5f2fa42a7af8c72505053c32d"
format_stamp: "Formatted at 2026-08-09 16:20:41 on dist-test-slave-s11t"
server_key: "d72d698f408ac67682e234b338bd4cb9"
server_key_iv: "a75f6b15d5eb3507c99dbb1caaa3ae9d"
server_key_version: "encryptionkey@0"
I20260809 16:20:41.687412 28890 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786292417722232-21710-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786292417722232-21710-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786292417722232-21710-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:20:41.690899 28890 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:20:41.691155 28890 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:20:41.691301 28890 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:20:41.695355 28890 rpc_server.cc:307] RPC server started. Bound to: 127.21.51.190:42739
I20260809 16:20:41.695390 29009 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.51.190:42739 every 8 connection(s)
I20260809 16:20:41.695724 28890 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786292417722232-21710-0/minicluster-data/master-0/data/info.pb
I20260809 16:20:41.696372 29010 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:20:41.699096 29010 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7c2c6ec5f2fa42a7af8c72505053c32d: Bootstrap starting.
I20260809 16:20:41.699671 29010 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7c2c6ec5f2fa42a7af8c72505053c32d: Neither blocks nor log segments found. Creating new log.
I20260809 16:20:41.699977 29010 log.cc:824] T 00000000000000000000000000000000 P 7c2c6ec5f2fa42a7af8c72505053c32d: Log is configured to *not* fsync() on all Append() calls
I20260809 16:20:41.700767 29010 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7c2c6ec5f2fa42a7af8c72505053c32d: No bootstrap required, opened a new log
I20260809 16:20:41.702328 29010 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7c2c6ec5f2fa42a7af8c72505053c32d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c2c6ec5f2fa42a7af8c72505053c32d" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 42739 } }
I20260809 16:20:41.702461 29010 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7c2c6ec5f2fa42a7af8c72505053c32d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:20:41.702507 29010 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7c2c6ec5f2fa42a7af8c72505053c32d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c2c6ec5f2fa42a7af8c72505053c32d, State: Initialized, Role: FOLLOWER
I20260809 16:20:41.702678 29010 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7c2c6ec5f2fa42a7af8c72505053c32d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c2c6ec5f2fa42a7af8c72505053c32d" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 42739 } }
I20260809 16:20:41.702776 29010 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7c2c6ec5f2fa42a7af8c72505053c32d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 16:20:41.702854 29010 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7c2c6ec5f2fa42a7af8c72505053c32d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 16:20:41.702911 29010 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7c2c6ec5f2fa42a7af8c72505053c32d [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:20:41.703465 29010 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7c2c6ec5f2fa42a7af8c72505053c32d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c2c6ec5f2fa42a7af8c72505053c32d" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 42739 } }
I20260809 16:20:41.703580 29010 leader_election.cc:302] T 00000000000000000000000000000000 P 7c2c6ec5f2fa42a7af8c72505053c32d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7c2c6ec5f2fa42a7af8c72505053c32d; no voters:
I20260809 16:20:41.703763 29010 leader_election.cc:288] T 00000000000000000000000000000000 P 7c2c6ec5f2fa42a7af8c72505053c32d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 16:20:41.703792 29016 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7c2c6ec5f2fa42a7af8c72505053c32d [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:20:41.703974 29016 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7c2c6ec5f2fa42a7af8c72505053c32d [term 1 LEADER]: Becoming Leader. State: Replica: 7c2c6ec5f2fa42a7af8c72505053c32d, State: Running, Role: LEADER
I20260809 16:20:41.704010 29010 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7c2c6ec5f2fa42a7af8c72505053c32d [sys.catalog]: configured and running, proceeding with master startup.
I20260809 16:20:41.704068 29016 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7c2c6ec5f2fa42a7af8c72505053c32d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c2c6ec5f2fa42a7af8c72505053c32d" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 42739 } }
I20260809 16:20:41.704757 21710 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrNqUCn/build/release/bin/kudu as pid 28890
I20260809 16:20:41.704734 29018 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7c2c6ec5f2fa42a7af8c72505053c32d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7c2c6ec5f2fa42a7af8c72505053c32d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c2c6ec5f2fa42a7af8c72505053c32d" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 42739 } } }
I20260809 16:20:41.704823 29018 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7c2c6ec5f2fa42a7af8c72505053c32d [sys.catalog]: This master's current role is: LEADER
I20260809 16:20:41.704870 21710 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786292417722232-21710-0/minicluster-data/master-0/wal/instance
I20260809 16:20:41.705087 21710 external_mini_cluster.cc:1428] Setting key fd0743a56aa0ec5ca8c81e9912976693
I20260809 16:20:41.705577 29019 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7c2c6ec5f2fa42a7af8c72505053c32d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7c2c6ec5f2fa42a7af8c72505053c32d. Latest consensus state: current_term: 1 leader_uuid: "7c2c6ec5f2fa42a7af8c72505053c32d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c2c6ec5f2fa42a7af8c72505053c32d" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 42739 } } }
I20260809 16:20:41.705686 29019 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7c2c6ec5f2fa42a7af8c72505053c32d [sys.catalog]: This master's current role is: LEADER
I20260809 16:20:41.705586 29033 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 16:20:41.706336 29033 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 16:20:41.707801 29044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:41.705698 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55528 (local address 127.21.51.190:42739)
0809 16:20:41.705925 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0809 16:20:41.705934 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 16:20:41.706148 (+ 214us) server_negotiation.cc:407] Connection header received
0809 16:20:41.706274 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:20:41.706281 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:20:41.706327 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:20:41.706512 (+ 185us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:20:41.707228 (+ 716us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:20:41.707231 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:20:41.707231 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 16:20:41.707235 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:20:41.707273 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 16:20:41.707286 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:20:41.707352 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0809 16:20:41.707569 (+ 217us) server_negotiation.cc:299] Negotiation successful
0809 16:20:41.707627 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":150,"thread_start_us":83,"threads_started":1}
I20260809 16:20:41.708194 29033 catalog_manager.cc:1347] Generated new cluster ID: b908354ad39940fabc63e744f04048e6
I20260809 16:20:41.708263 29033 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 16:20:41.721637 29033 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 16:20:41.721844 29033 catalog_manager.cc:1504] Loading token signing keys...
I20260809 16:20:41.725178 29033 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7c2c6ec5f2fa42a7af8c72505053c32d: Generated new TSK 0
I20260809 16:20:41.725374 29033 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 16:20:41.728456 21710 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrNqUCn/build/release/bin/kudu
/tmp/dist-test-taskrNqUCn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786292417722232-21710-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786292417722232-21710-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786292417722232-21710-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786292417722232-21710-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.51.129:0
--local_ip_for_outbound_sockets=127.21.51.129
--webserver_interface=127.21.51.129
--webserver_port=0
--tserver_master_addrs=127.21.51.190:42739
--builtin_ntp_servers=127.21.51.148:34031
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260809 16:20:41.812458 29050 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:20:41.812724 29050 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:20:41.812777 29050 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:20:41.814008 29050 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:20:41.814066 29050 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:20:41.814108 29050 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.51.129
I20260809 16:20:41.815496 29050 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.51.148:34031
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786292417722232-21710-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786292417722232-21710-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.51.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786292417722232-21710-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.51.129
--webserver_port=0
--tserver_master_addrs=127.21.51.190:42739
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29050
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.51.129
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786292417722232-21710-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:20:41.815760 29050 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:20:41.816066 29050 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:20:41.821194 29059 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:20:41.821254 29061 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:20:41.821403 29050 server_base.cc:1034] running on GCE node
W20260809 16:20:41.821233 29065 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:20:41.821792 29050 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:20:41.822139 29050 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:20:41.823333 29050 hybrid_clock.cc:648] HybridClock initialized: now 1786292441823321 us; error 40 us; skew 500 ppm
I20260809 16:20:41.824724 29050 webserver.cc:460] Webserver started at http://127.21.51.129:33637/ using document root <none> and password file <none>
I20260809 16:20:41.824983 29050 fs_manager.cc:359] Metadata directory not provided
I20260809 16:20:41.825040 29050 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:20:41.825161 29050 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:20:41.826529 29050 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786292417722232-21710-0/minicluster-data/ts-0/data/instance:
uuid: "1ed01a8ae0c3497bbb997a6f77fc46bb"
format_stamp: "Formatted at 2026-08-09 16:20:41 on dist-test-slave-s11t"
server_key: "c2fbaad07a51023e0b8496e76c83d55d"
server_key_iv: "19ce3c3a61691e4bd634beba24f28e97"
server_key_version: "encryptionkey@0"
I20260809 16:20:41.826930 29050 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786292417722232-21710-0/minicluster-data/ts-0/wal/instance:
uuid: "1ed01a8ae0c3497bbb997a6f77fc46bb"
format_stamp: "Formatted at 2026-08-09 16:20:41 on dist-test-slave-s11t"
server_key: "c2fbaad07a51023e0b8496e76c83d55d"
server_key_iv: "19ce3c3a61691e4bd634beba24f28e97"
server_key_version: "encryptionkey@0"
I20260809 16:20:41.828464 29050 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 16:20:41.829411 29071 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:20:41.829622 29050 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:20:41.829718 29050 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786292417722232-21710-0/minicluster-data/ts-0/data,/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786292417722232-21710-0/minicluster-data/ts-0/wal
uuid: "1ed01a8ae0c3497bbb997a6f77fc46bb"
format_stamp: "Formatted at 2026-08-09 16:20:41 on dist-test-slave-s11t"
server_key: "c2fbaad07a51023e0b8496e76c83d55d"
server_key_iv: "19ce3c3a61691e4bd634beba24f28e97"
server_key_version: "encryptionkey@0"
I20260809 16:20:41.829806 29050 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786292417722232-21710-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786292417722232-21710-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786292417722232-21710-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:20:41.834236 29050 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:20:41.834465 29050 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:20:41.834559 29050 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:20:41.834715 29050 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:20:41.834968 29050 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:20:41.834998 29050 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:20:41.835021 29050 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:20:41.835034 29050 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:20:41.840740 29050 rpc_server.cc:307] RPC server started. Bound to: 127.21.51.129:35793
I20260809 16:20:41.840794 29243 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.51.129:35793 every 8 connection(s)
I20260809 16:20:41.841094 29050 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786292417722232-21710-0/minicluster-data/ts-0/data/info.pb
I20260809 16:20:41.844040 29044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:41.841767 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.51.129:52625 (local address 127.21.51.190:42739)
0809 16:20:41.841810 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0809 16:20:41.841813 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 16:20:41.842065 (+ 252us) server_negotiation.cc:407] Connection header received
0809 16:20:41.842557 (+ 492us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:20:41.842558 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:20:41.842566 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:20:41.842636 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:20:41.843753 (+ 1117us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:20:41.843754 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:20:41.843755 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 16:20:41.843759 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:20:41.843775 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 16:20:41.843780 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:20:41.843854 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0809 16:20:41.843951 (+ 97us) server_negotiation.cc:299] Negotiation successful
0809 16:20:41.843969 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260809 16:20:41.844151 21710 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrNqUCn/build/release/bin/kudu as pid 29050
I20260809 16:20:41.844244 21710 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786292417722232-21710-0/minicluster-data/ts-0/wal/instance
I20260809 16:20:41.844266 29249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:41.841738 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:42739 (local address 127.21.51.129:52625)
0809 16:20:41.841952 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0809 16:20:41.841983 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0809 16:20:41.842453 (+ 470us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:20:41.842719 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:20:41.842725 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:20:41.842739 (+ 14us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:20:41.843606 (+ 867us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 16:20:41.843606 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:20:41.843631 (+ 25us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 16:20:41.843656 (+ 25us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:20:41.843853 (+ 197us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:20:41.843860 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:20:41.843863 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 16:20:41.843980 (+ 117us) client_negotiation.cc:240] Negotiation successful
0809 16:20:41.844039 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":105,"threads_started":1}
I20260809 16:20:41.844480 21710 external_mini_cluster.cc:1428] Setting key e8d180fa507b281421aebccd46a9ff77
I20260809 16:20:41.844802 29245 heartbeater.cc:347] Connected to a master server at 127.21.51.190:42739
I20260809 16:20:41.844894 29245 heartbeater.cc:464] Registering TS with master...
I20260809 16:20:41.845027 29245 heartbeater.cc:511] Master 127.21.51.190:42739 requested a full tablet report, sending...
I20260809 16:20:41.845561 21710 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrNqUCn/build/release/bin/kudu
/tmp/dist-test-taskrNqUCn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786292417722232-21710-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786292417722232-21710-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786292417722232-21710-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786292417722232-21710-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.51.130:0
--local_ip_for_outbound_sockets=127.21.51.130
--webserver_interface=127.21.51.130
--webserver_port=0
--tserver_master_addrs=127.21.51.190:42739
--builtin_ntp_servers=127.21.51.148:34031
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260809 16:20:41.845623 28949 ts_manager.cc:194] Registered new tserver with Master: 1ed01a8ae0c3497bbb997a6f77fc46bb (127.21.51.129:35793)
W20260809 16:20:41.927342 29250 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:20:41.927541 29250 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:20:41.927562 29250 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:20:41.928728 29250 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:20:41.928769 29250 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:20:41.928794 29250 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.51.130
I20260809 16:20:41.930167 29250 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.51.148:34031
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786292417722232-21710-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786292417722232-21710-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.51.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786292417722232-21710-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.51.130
--webserver_port=0
--tserver_master_addrs=127.21.51.190:42739
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29250
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.51.130
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786292417722232-21710-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:20:41.930413 29250 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:20:41.930722 29250 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:20:41.935904 29256 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:20:41.935954 29259 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:20:41.935918 29257 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:20:41.936237 29250 server_base.cc:1034] running on GCE node
I20260809 16:20:41.936491 29250 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:20:41.936770 29250 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:20:41.937916 29250 hybrid_clock.cc:648] HybridClock initialized: now 1786292441937896 us; error 37 us; skew 500 ppm
I20260809 16:20:41.939150 29250 webserver.cc:460] Webserver started at http://127.21.51.130:41313/ using document root <none> and password file <none>
I20260809 16:20:41.939414 29250 fs_manager.cc:359] Metadata directory not provided
I20260809 16:20:41.939460 29250 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:20:41.939630 29250 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:20:41.940810 29250 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786292417722232-21710-0/minicluster-data/ts-1/data/instance:
uuid: "18290c4f39d74d8bb5275a1f9d5f183b"
format_stamp: "Formatted at 2026-08-09 16:20:41 on dist-test-slave-s11t"
server_key: "1120cc507af99279b8f27d13d744f976"
server_key_iv: "a8075e14161d2efddb1ad0d938981b6c"
server_key_version: "encryptionkey@0"
I20260809 16:20:41.941175 29250 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786292417722232-21710-0/minicluster-data/ts-1/wal/instance:
uuid: "18290c4f39d74d8bb5275a1f9d5f183b"
format_stamp: "Formatted at 2026-08-09 16:20:41 on dist-test-slave-s11t"
server_key: "1120cc507af99279b8f27d13d744f976"
server_key_iv: "a8075e14161d2efddb1ad0d938981b6c"
server_key_version: "encryptionkey@0"
I20260809 16:20:41.942489 29250 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 16:20:41.943228 29267 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:20:41.943389 29250 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 16:20:41.943472 29250 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786292417722232-21710-0/minicluster-data/ts-1/data,/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786292417722232-21710-0/minicluster-data/ts-1/wal
uuid: "18290c4f39d74d8bb5275a1f9d5f183b"
format_stamp: "Formatted at 2026-08-09 16:20:41 on dist-test-slave-s11t"
server_key: "1120cc507af99279b8f27d13d744f976"
server_key_iv: "a8075e14161d2efddb1ad0d938981b6c"
server_key_version: "encryptionkey@0"
I20260809 16:20:41.943526 29250 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786292417722232-21710-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786292417722232-21710-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786292417722232-21710-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:20:41.946837 29250 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:20:41.947041 29250 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:20:41.947131 29250 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:20:41.947288 29250 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:20:41.947527 29250 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:20:41.947557 29250 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:20:41.947578 29250 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:20:41.947590 29250 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:20:41.953256 29250 rpc_server.cc:307] RPC server started. Bound to: 127.21.51.130:44979
I20260809 16:20:41.953336 29446 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.51.130:44979 every 8 connection(s)
I20260809 16:20:41.953564 29250 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786292417722232-21710-0/minicluster-data/ts-1/data/info.pb
I20260809 16:20:41.956526 29044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:41.954271 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.51.130:38271 (local address 127.21.51.190:42739)
0809 16:20:41.954320 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0809 16:20:41.954322 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:20:41.954534 (+ 212us) server_negotiation.cc:407] Connection header received
0809 16:20:41.955043 (+ 509us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:20:41.955045 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:20:41.955052 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:20:41.955112 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:20:41.956248 (+ 1136us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:20:41.956250 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:20:41.956250 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 16:20:41.956255 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:20:41.956275 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 16:20:41.956280 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:20:41.956354 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0809 16:20:41.956441 (+ 87us) server_negotiation.cc:299] Negotiation successful
0809 16:20:41.956456 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260809 16:20:41.956775 29454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:41.954239 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:42739 (local address 127.21.51.130:38271)
0809 16:20:41.954409 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0809 16:20:41.954442 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0809 16:20:41.954929 (+ 487us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:20:41.955179 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:20:41.955182 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:20:41.955192 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:20:41.956083 (+ 891us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 16:20:41.956084 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:20:41.956113 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 16:20:41.956140 (+ 27us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:20:41.956363 (+ 223us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:20:41.956369 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:20:41.956373 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 16:20:41.956440 (+ 67us) client_negotiation.cc:240] Negotiation successful
0809 16:20:41.956503 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":66,"threads_started":1}
I20260809 16:20:41.957324 29450 heartbeater.cc:347] Connected to a master server at 127.21.51.190:42739
I20260809 16:20:41.957425 29450 heartbeater.cc:464] Registering TS with master...
I20260809 16:20:41.957581 29450 heartbeater.cc:511] Master 127.21.51.190:42739 requested a full tablet report, sending...
I20260809 16:20:41.957976 28949 ts_manager.cc:194] Registered new tserver with Master: 18290c4f39d74d8bb5275a1f9d5f183b (127.21.51.130:44979)
I20260809 16:20:41.961002 21710 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrNqUCn/build/release/bin/kudu as pid 29250
I20260809 16:20:41.961092 21710 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786292417722232-21710-0/minicluster-data/ts-1/wal/instance
I20260809 16:20:41.961282 21710 external_mini_cluster.cc:1428] Setting key 3b0ae67a50d3b85392d85739fd6ed35c
I20260809 16:20:41.962347 21710 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrNqUCn/build/release/bin/kudu
/tmp/dist-test-taskrNqUCn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786292417722232-21710-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786292417722232-21710-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786292417722232-21710-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786292417722232-21710-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.51.131:0
--local_ip_for_outbound_sockets=127.21.51.131
--webserver_interface=127.21.51.131
--webserver_port=0
--tserver_master_addrs=127.21.51.190:42739
--builtin_ntp_servers=127.21.51.148:34031
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260809 16:20:42.047714 29455 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:20:42.047943 29455 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:20:42.047974 29455 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:20:42.049326 29455 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:20:42.049371 29455 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:20:42.049398 29455 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.51.131
I20260809 16:20:42.050875 29455 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.51.148:34031
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786292417722232-21710-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786292417722232-21710-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.51.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786292417722232-21710-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.51.131
--webserver_port=0
--tserver_master_addrs=127.21.51.190:42739
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29455
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.51.131
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786292417722232-21710-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:20:42.051141 29455 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:20:42.051465 29455 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:20:42.056999 29463 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:20:42.057008 29465 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:20:42.057008 29462 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:20:42.057338 29455 server_base.cc:1034] running on GCE node
I20260809 16:20:42.057564 29455 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:20:42.057754 29455 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:20:42.058928 29455 hybrid_clock.cc:648] HybridClock initialized: now 1786292442058923 us; error 37 us; skew 500 ppm
I20260809 16:20:42.060122 29455 webserver.cc:460] Webserver started at http://127.21.51.131:45619/ using document root <none> and password file <none>
I20260809 16:20:42.060362 29455 fs_manager.cc:359] Metadata directory not provided
I20260809 16:20:42.060436 29455 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:20:42.060593 29455 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:20:42.061686 29455 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786292417722232-21710-0/minicluster-data/ts-2/data/instance:
uuid: "d6568f7cf549467f8a0fde547cd97a6e"
format_stamp: "Formatted at 2026-08-09 16:20:42 on dist-test-slave-s11t"
server_key: "029092997316c6da5eae2163f0d3395f"
server_key_iv: "cbdb090542d9b1a28b7451b65ffba7e8"
server_key_version: "encryptionkey@0"
I20260809 16:20:42.062067 29455 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786292417722232-21710-0/minicluster-data/ts-2/wal/instance:
uuid: "d6568f7cf549467f8a0fde547cd97a6e"
format_stamp: "Formatted at 2026-08-09 16:20:42 on dist-test-slave-s11t"
server_key: "029092997316c6da5eae2163f0d3395f"
server_key_iv: "cbdb090542d9b1a28b7451b65ffba7e8"
server_key_version: "encryptionkey@0"
I20260809 16:20:42.063424 29455 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260809 16:20:42.064217 29474 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:20:42.064419 29455 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 16:20:42.064520 29455 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786292417722232-21710-0/minicluster-data/ts-2/data,/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786292417722232-21710-0/minicluster-data/ts-2/wal
uuid: "d6568f7cf549467f8a0fde547cd97a6e"
format_stamp: "Formatted at 2026-08-09 16:20:42 on dist-test-slave-s11t"
server_key: "029092997316c6da5eae2163f0d3395f"
server_key_iv: "cbdb090542d9b1a28b7451b65ffba7e8"
server_key_version: "encryptionkey@0"
I20260809 16:20:42.064603 29455 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786292417722232-21710-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786292417722232-21710-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786292417722232-21710-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:20:42.067992 29455 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:20:42.068259 29455 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:20:42.068406 29455 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:20:42.068621 29455 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:20:42.068969 29455 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:20:42.069026 29455 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:20:42.069067 29455 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:20:42.069093 29455 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:20:42.074815 29455 rpc_server.cc:307] RPC server started. Bound to: 127.21.51.131:46375
I20260809 16:20:42.074857 29649 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.51.131:46375 every 8 connection(s)
I20260809 16:20:42.075176 29455 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786292417722232-21710-0/minicluster-data/ts-2/data/info.pb
I20260809 16:20:42.077988 21710 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrNqUCn/build/release/bin/kudu as pid 29455
I20260809 16:20:42.078116 21710 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786292417722232-21710-0/minicluster-data/ts-2/wal/instance
I20260809 16:20:42.078269 29044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:42.075900 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.51.131:38573 (local address 127.21.51.190:42739)
0809 16:20:42.075972 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0809 16:20:42.075974 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:20:42.076194 (+ 220us) server_negotiation.cc:407] Connection header received
0809 16:20:42.076641 (+ 447us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:20:42.076642 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:20:42.076647 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:20:42.076793 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:20:42.077949 (+ 1156us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:20:42.077950 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:20:42.077950 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 16:20:42.077954 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:20:42.077970 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 16:20:42.077974 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:20:42.078025 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0809 16:20:42.078180 (+ 155us) server_negotiation.cc:299] Negotiation successful
0809 16:20:42.078198 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260809 16:20:42.078334 21710 external_mini_cluster.cc:1428] Setting key 28bab8b3593cecf074840b49daf91375
I20260809 16:20:42.078454 29653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:42.075886 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:42739 (local address 127.21.51.131:38573)
0809 16:20:42.076079 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0809 16:20:42.076110 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0809 16:20:42.076556 (+ 446us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:20:42.076864 (+ 308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:20:42.076867 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:20:42.076875 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:20:42.077725 (+ 850us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 16:20:42.077726 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:20:42.077752 (+ 26us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 16:20:42.077845 (+ 93us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:20:42.078040 (+ 195us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:20:42.078045 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:20:42.078049 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 16:20:42.078167 (+ 118us) client_negotiation.cc:240] Negotiation successful
0809 16:20:42.078232 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":99,"threads_started":1}
I20260809 16:20:42.078908 29650 heartbeater.cc:347] Connected to a master server at 127.21.51.190:42739
I20260809 16:20:42.078998 29650 heartbeater.cc:464] Registering TS with master...
I20260809 16:20:42.079106 29650 heartbeater.cc:511] Master 127.21.51.190:42739 requested a full tablet report, sending...
I20260809 16:20:42.079466 28949 ts_manager.cc:194] Registered new tserver with Master: d6568f7cf549467f8a0fde547cd97a6e (127.21.51.131:46375)
I20260809 16:20:42.090234 21710 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 16:20:42.092797 29044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:42.091595 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55538 (local address 127.21.51.190:42739)
0809 16:20:42.091644 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0809 16:20:42.091646 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:20:42.091900 (+ 254us) server_negotiation.cc:407] Connection header received
0809 16:20:42.091940 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:20:42.091941 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:20:42.091946 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:20:42.091995 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:20:42.092532 (+ 537us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:20:42.092534 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:20:42.092534 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 16:20:42.092538 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:20:42.092555 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 16:20:42.092559 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:20:42.092619 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0809 16:20:42.092658 (+ 39us) server_negotiation.cc:299] Negotiation successful
0809 16:20:42.092670 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260809 16:20:42.094527 28949 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:55538:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 16:20:42.095957 28949 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 16:20:42.099668 29668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:42.098047 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.130:44979 (local address 127.0.0.1:52684)
0809 16:20:42.098388 (+ 341us) negotiation.cc:107] Waiting for socket to connect
0809 16:20:42.098433 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0809 16:20:42.098492 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:20:42.098721 (+ 229us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:20:42.098726 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:20:42.098731 (+ 5us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:20:42.099237 (+ 506us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 16:20:42.099238 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:20:42.099255 (+ 17us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 16:20:42.099277 (+ 22us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:20:42.099523 (+ 246us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:20:42.099529 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:20:42.099531 (+ 2us) client_negotiation.cc:769] Sending connection context
0809 16:20:42.099576 (+ 45us) client_negotiation.cc:240] Negotiation successful
0809 16:20:42.099589 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":314,"thread_start_us":81,"threads_started":1}
I20260809 16:20:42.099998 29670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:42.098165 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52684 (local address 127.21.51.130:44979)
0809 16:20:42.098580 (+ 415us) server_negotiation.cc:206] Beginning negotiation
0809 16:20:42.098585 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 16:20:42.098597 (+ 12us) server_negotiation.cc:407] Connection header received
0809 16:20:42.098626 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:20:42.098628 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:20:42.098639 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:20:42.098710 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:20:42.099372 (+ 662us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:20:42.099381 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:20:42.099381 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 16:20:42.099385 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:20:42.099420 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 16:20:42.099446 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:20:42.099877 (+ 431us) server_negotiation.cc:1035] Waiting for connection context
0809 16:20:42.099904 (+ 27us) server_negotiation.cc:299] Negotiation successful
0809 16:20:42.099920 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":372,"thread_start_us":117,"threads_started":1}
I20260809 16:20:42.100004 29673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:42.098273 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.131:46375 (local address 127.0.0.1:33224)
0809 16:20:42.098583 (+ 310us) negotiation.cc:107] Waiting for socket to connect
0809 16:20:42.098591 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0809 16:20:42.098637 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:20:42.099104 (+ 467us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:20:42.099105 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:20:42.099108 (+ 3us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:20:42.099590 (+ 482us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 16:20:42.099591 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:20:42.099598 (+ 7us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 16:20:42.099610 (+ 12us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:20:42.099875 (+ 265us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:20:42.099876 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:20:42.099877 (+ 1us) client_negotiation.cc:769] Sending connection context
0809 16:20:42.099926 (+ 49us) client_negotiation.cc:240] Negotiation successful
0809 16:20:42.099940 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":294,"thread_start_us":78,"threads_started":1}
I20260809 16:20:42.100023 29674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:42.098376 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33224 (local address 127.21.51.131:46375)
0809 16:20:42.098557 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0809 16:20:42.098562 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 16:20:42.098633 (+ 71us) server_negotiation.cc:407] Connection header received
0809 16:20:42.098955 (+ 322us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:20:42.098960 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:20:42.098970 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:20:42.099045 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:20:42.099681 (+ 636us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:20:42.099687 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:20:42.099688 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 16:20:42.099691 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:20:42.099749 (+ 58us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 16:20:42.099776 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:20:42.099832 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0809 16:20:42.099935 (+ 103us) server_negotiation.cc:299] Negotiation successful
0809 16:20:42.099950 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":141,"thread_start_us":85,"threads_started":1}
I20260809 16:20:42.100497 29669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:42.098047 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.129:35793 (local address 127.0.0.1:52974)
0809 16:20:42.098413 (+ 366us) negotiation.cc:107] Waiting for socket to connect
0809 16:20:42.098448 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0809 16:20:42.098516 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:20:42.099474 (+ 958us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:20:42.099475 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:20:42.099478 (+ 3us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:20:42.100127 (+ 649us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 16:20:42.100128 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:20:42.100134 (+ 6us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 16:20:42.100147 (+ 13us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:20:42.100359 (+ 212us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:20:42.100360 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:20:42.100360 (+ 0us) client_negotiation.cc:769] Sending connection context
0809 16:20:42.100424 (+ 64us) client_negotiation.cc:240] Negotiation successful
0809 16:20:42.100436 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":349,"mutex_wait_us":2,"thread_start_us":65,"threads_started":1}
I20260809 16:20:42.100554 29671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:42.098232 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52974 (local address 127.21.51.129:35793)
0809 16:20:42.099125 (+ 893us) server_negotiation.cc:206] Beginning negotiation
0809 16:20:42.099129 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 16:20:42.099138 (+ 9us) server_negotiation.cc:407] Connection header received
0809 16:20:42.099163 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:20:42.099166 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:20:42.099175 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:20:42.099257 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:20:42.100230 (+ 973us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:20:42.100238 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:20:42.100238 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 16:20:42.100243 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:20:42.100283 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 16:20:42.100303 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:20:42.100346 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0809 16:20:42.100454 (+ 108us) server_negotiation.cc:299] Negotiation successful
0809 16:20:42.100471 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":856,"thread_start_us":85,"threads_started":1}
I20260809 16:20:42.100879 29542 tablet_service.cc:1467] Processing CreateTablet for tablet 4b65462f4d1d468392ca83a637347792 (DEFAULT_TABLE table=test-table [id=566b23ccff5d417b98844e4ad50cc11b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:20:42.100980 29337 tablet_service.cc:1467] Processing CreateTablet for tablet 4b65462f4d1d468392ca83a637347792 (DEFAULT_TABLE table=test-table [id=566b23ccff5d417b98844e4ad50cc11b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:20:42.101239 29542 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4b65462f4d1d468392ca83a637347792. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:20:42.101310 29337 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4b65462f4d1d468392ca83a637347792. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:20:42.101243 29146 tablet_service.cc:1467] Processing CreateTablet for tablet 4b65462f4d1d468392ca83a637347792 (DEFAULT_TABLE table=test-table [id=566b23ccff5d417b98844e4ad50cc11b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:20:42.101543 29146 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4b65462f4d1d468392ca83a637347792. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:20:42.103855 29676 tablet_bootstrap.cc:492] T 4b65462f4d1d468392ca83a637347792 P 18290c4f39d74d8bb5275a1f9d5f183b: Bootstrap starting.
I20260809 16:20:42.104568 29676 tablet_bootstrap.cc:654] T 4b65462f4d1d468392ca83a637347792 P 18290c4f39d74d8bb5275a1f9d5f183b: Neither blocks nor log segments found. Creating new log.
I20260809 16:20:42.104817 29678 tablet_bootstrap.cc:492] T 4b65462f4d1d468392ca83a637347792 P 1ed01a8ae0c3497bbb997a6f77fc46bb: Bootstrap starting.
I20260809 16:20:42.104960 29676 log.cc:824] T 4b65462f4d1d468392ca83a637347792 P 18290c4f39d74d8bb5275a1f9d5f183b: Log is configured to *not* fsync() on all Append() calls
I20260809 16:20:42.105513 29678 tablet_bootstrap.cc:654] T 4b65462f4d1d468392ca83a637347792 P 1ed01a8ae0c3497bbb997a6f77fc46bb: Neither blocks nor log segments found. Creating new log.
I20260809 16:20:42.105758 29676 tablet_bootstrap.cc:492] T 4b65462f4d1d468392ca83a637347792 P 18290c4f39d74d8bb5275a1f9d5f183b: No bootstrap required, opened a new log
I20260809 16:20:42.105839 29676 ts_tablet_manager.cc:1397] T 4b65462f4d1d468392ca83a637347792 P 18290c4f39d74d8bb5275a1f9d5f183b: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 16:20:42.105825 29678 log.cc:824] T 4b65462f4d1d468392ca83a637347792 P 1ed01a8ae0c3497bbb997a6f77fc46bb: Log is configured to *not* fsync() on all Append() calls
I20260809 16:20:42.106086 29677 tablet_bootstrap.cc:492] T 4b65462f4d1d468392ca83a637347792 P d6568f7cf549467f8a0fde547cd97a6e: Bootstrap starting.
I20260809 16:20:42.106597 29678 tablet_bootstrap.cc:492] T 4b65462f4d1d468392ca83a637347792 P 1ed01a8ae0c3497bbb997a6f77fc46bb: No bootstrap required, opened a new log
I20260809 16:20:42.106679 29678 ts_tablet_manager.cc:1397] T 4b65462f4d1d468392ca83a637347792 P 1ed01a8ae0c3497bbb997a6f77fc46bb: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260809 16:20:42.106711 29677 tablet_bootstrap.cc:654] T 4b65462f4d1d468392ca83a637347792 P d6568f7cf549467f8a0fde547cd97a6e: Neither blocks nor log segments found. Creating new log.
I20260809 16:20:42.107028 29677 log.cc:824] T 4b65462f4d1d468392ca83a637347792 P d6568f7cf549467f8a0fde547cd97a6e: Log is configured to *not* fsync() on all Append() calls
I20260809 16:20:42.107815 29677 tablet_bootstrap.cc:492] T 4b65462f4d1d468392ca83a637347792 P d6568f7cf549467f8a0fde547cd97a6e: No bootstrap required, opened a new log
I20260809 16:20:42.107903 29677 ts_tablet_manager.cc:1397] T 4b65462f4d1d468392ca83a637347792 P d6568f7cf549467f8a0fde547cd97a6e: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260809 16:20:42.108594 29676 raft_consensus.cc:348] T 4b65462f4d1d468392ca83a637347792 P 18290c4f39d74d8bb5275a1f9d5f183b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18290c4f39d74d8bb5275a1f9d5f183b" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 44979 } } peers { permanent_uuid: "1ed01a8ae0c3497bbb997a6f77fc46bb" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 35793 } } peers { permanent_uuid: "d6568f7cf549467f8a0fde547cd97a6e" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 46375 } }
I20260809 16:20:42.108790 29676 raft_consensus.cc:374] T 4b65462f4d1d468392ca83a637347792 P 18290c4f39d74d8bb5275a1f9d5f183b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:20:42.108837 29676 raft_consensus.cc:729] T 4b65462f4d1d468392ca83a637347792 P 18290c4f39d74d8bb5275a1f9d5f183b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 18290c4f39d74d8bb5275a1f9d5f183b, State: Initialized, Role: FOLLOWER
I20260809 16:20:42.108841 29678 raft_consensus.cc:348] T 4b65462f4d1d468392ca83a637347792 P 1ed01a8ae0c3497bbb997a6f77fc46bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18290c4f39d74d8bb5275a1f9d5f183b" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 44979 } } peers { permanent_uuid: "1ed01a8ae0c3497bbb997a6f77fc46bb" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 35793 } } peers { permanent_uuid: "d6568f7cf549467f8a0fde547cd97a6e" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 46375 } }
I20260809 16:20:42.108953 29676 consensus_queue.cc:260] T 4b65462f4d1d468392ca83a637347792 P 18290c4f39d74d8bb5275a1f9d5f183b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18290c4f39d74d8bb5275a1f9d5f183b" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 44979 } } peers { permanent_uuid: "1ed01a8ae0c3497bbb997a6f77fc46bb" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 35793 } } peers { permanent_uuid: "d6568f7cf549467f8a0fde547cd97a6e" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 46375 } }
I20260809 16:20:42.109042 29678 raft_consensus.cc:374] T 4b65462f4d1d468392ca83a637347792 P 1ed01a8ae0c3497bbb997a6f77fc46bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:20:42.109087 29678 raft_consensus.cc:729] T 4b65462f4d1d468392ca83a637347792 P 1ed01a8ae0c3497bbb997a6f77fc46bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1ed01a8ae0c3497bbb997a6f77fc46bb, State: Initialized, Role: FOLLOWER
I20260809 16:20:42.109213 29676 ts_tablet_manager.cc:1428] T 4b65462f4d1d468392ca83a637347792 P 18290c4f39d74d8bb5275a1f9d5f183b: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260809 16:20:42.109192 29678 consensus_queue.cc:260] T 4b65462f4d1d468392ca83a637347792 P 1ed01a8ae0c3497bbb997a6f77fc46bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18290c4f39d74d8bb5275a1f9d5f183b" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 44979 } } peers { permanent_uuid: "1ed01a8ae0c3497bbb997a6f77fc46bb" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 35793 } } peers { permanent_uuid: "d6568f7cf549467f8a0fde547cd97a6e" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 46375 } }
I20260809 16:20:42.109347 29450 heartbeater.cc:503] Master 127.21.51.190:42739 was elected leader, sending a full tablet report...
I20260809 16:20:42.109529 29245 heartbeater.cc:503] Master 127.21.51.190:42739 was elected leader, sending a full tablet report...
I20260809 16:20:42.109591 29678 ts_tablet_manager.cc:1428] T 4b65462f4d1d468392ca83a637347792 P 1ed01a8ae0c3497bbb997a6f77fc46bb: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260809 16:20:42.110901 29677 raft_consensus.cc:348] T 4b65462f4d1d468392ca83a637347792 P d6568f7cf549467f8a0fde547cd97a6e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18290c4f39d74d8bb5275a1f9d5f183b" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 44979 } } peers { permanent_uuid: "1ed01a8ae0c3497bbb997a6f77fc46bb" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 35793 } } peers { permanent_uuid: "d6568f7cf549467f8a0fde547cd97a6e" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 46375 } }
I20260809 16:20:42.111019 29677 raft_consensus.cc:374] T 4b65462f4d1d468392ca83a637347792 P d6568f7cf549467f8a0fde547cd97a6e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:20:42.111040 29677 raft_consensus.cc:729] T 4b65462f4d1d468392ca83a637347792 P d6568f7cf549467f8a0fde547cd97a6e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6568f7cf549467f8a0fde547cd97a6e, State: Initialized, Role: FOLLOWER
I20260809 16:20:42.111143 29677 consensus_queue.cc:260] T 4b65462f4d1d468392ca83a637347792 P d6568f7cf549467f8a0fde547cd97a6e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18290c4f39d74d8bb5275a1f9d5f183b" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 44979 } } peers { permanent_uuid: "1ed01a8ae0c3497bbb997a6f77fc46bb" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 35793 } } peers { permanent_uuid: "d6568f7cf549467f8a0fde547cd97a6e" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 46375 } }
I20260809 16:20:42.111408 29677 ts_tablet_manager.cc:1428] T 4b65462f4d1d468392ca83a637347792 P d6568f7cf549467f8a0fde547cd97a6e: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260809 16:20:42.111425 29650 heartbeater.cc:503] Master 127.21.51.190:42739 was elected leader, sending a full tablet report...
I20260809 16:20:42.143464 29689 raft_consensus.cc:482] T 4b65462f4d1d468392ca83a637347792 P d6568f7cf549467f8a0fde547cd97a6e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 16:20:42.143573 29689 raft_consensus.cc:504] T 4b65462f4d1d468392ca83a637347792 P d6568f7cf549467f8a0fde547cd97a6e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18290c4f39d74d8bb5275a1f9d5f183b" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 44979 } } peers { permanent_uuid: "1ed01a8ae0c3497bbb997a6f77fc46bb" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 35793 } } peers { permanent_uuid: "d6568f7cf549467f8a0fde547cd97a6e" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 46375 } }
I20260809 16:20:42.143893 29689 leader_election.cc:288] T 4b65462f4d1d468392ca83a637347792 P d6568f7cf549467f8a0fde547cd97a6e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 18290c4f39d74d8bb5275a1f9d5f183b (127.21.51.130:44979), 1ed01a8ae0c3497bbb997a6f77fc46bb (127.21.51.129:35793)
I20260809 16:20:42.145242 29653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:42.144060 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.129:35793 (local address 127.21.51.131:57329)
0809 16:20:42.144133 (+ 73us) negotiation.cc:107] Waiting for socket to connect
0809 16:20:42.144144 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 16:20:42.144182 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:20:42.144327 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:20:42.144328 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:20:42.144331 (+ 3us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:20:42.144849 (+ 518us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 16:20:42.144850 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:20:42.144858 (+ 8us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 16:20:42.144873 (+ 15us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:20:42.145087 (+ 214us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:20:42.145088 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:20:42.145089 (+ 1us) client_negotiation.cc:769] Sending connection context
0809 16:20:42.145147 (+ 58us) client_negotiation.cc:240] Negotiation successful
0809 16:20:42.145162 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":54}
I20260809 16:20:42.145268 29671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:42.144129 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.51.131:57329 (local address 127.21.51.129:35793)
0809 16:20:42.144180 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0809 16:20:42.144182 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:20:42.144191 (+ 9us) server_negotiation.cc:407] Connection header received
0809 16:20:42.144242 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:20:42.144243 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:20:42.144248 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:20:42.144321 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:20:42.144963 (+ 642us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:20:42.144964 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:20:42.144964 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 16:20:42.144969 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:20:42.144990 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 16:20:42.144996 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:20:42.145061 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0809 16:20:42.145147 (+ 86us) server_negotiation.cc:299] Negotiation successful
0809 16:20:42.145166 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260809 16:20:42.145375 29670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:42.144109 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.51.131:37527 (local address 127.21.51.130:44979)
0809 16:20:42.144158 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0809 16:20:42.144160 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:20:42.144323 (+ 163us) server_negotiation.cc:407] Connection header received
0809 16:20:42.144411 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:20:42.144416 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:20:42.144421 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:20:42.144483 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:20:42.145124 (+ 641us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:20:42.145125 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:20:42.145125 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 16:20:42.145131 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:20:42.145150 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 16:20:42.145155 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:20:42.145211 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0809 16:20:42.145273 (+ 62us) server_negotiation.cc:299] Negotiation successful
0809 16:20:42.145287 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260809 16:20:42.145591 29690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:42.144074 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.130:44979 (local address 127.21.51.131:37527)
0809 16:20:42.144242 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0809 16:20:42.144252 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 16:20:42.144333 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:20:42.144503 (+ 170us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:20:42.144503 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:20:42.144505 (+ 2us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:20:42.145017 (+ 512us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 16:20:42.145017 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:20:42.145024 (+ 7us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 16:20:42.145037 (+ 13us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:20:42.145221 (+ 184us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:20:42.145221 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:20:42.145222 (+ 1us) client_negotiation.cc:769] Sending connection context
0809 16:20:42.145272 (+ 50us) client_negotiation.cc:240] Negotiation successful
0809 16:20:42.145286 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":121,"threads_started":1}
I20260809 16:20:42.145614 29171 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4b65462f4d1d468392ca83a637347792" candidate_uuid: "d6568f7cf549467f8a0fde547cd97a6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1ed01a8ae0c3497bbb997a6f77fc46bb" is_pre_election: true
I20260809 16:20:42.145761 29171 raft_consensus.cc:2454] T 4b65462f4d1d468392ca83a637347792 P 1ed01a8ae0c3497bbb997a6f77fc46bb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d6568f7cf549467f8a0fde547cd97a6e in term 0.
I20260809 16:20:42.145956 29367 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4b65462f4d1d468392ca83a637347792" candidate_uuid: "d6568f7cf549467f8a0fde547cd97a6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "18290c4f39d74d8bb5275a1f9d5f183b" is_pre_election: true
I20260809 16:20:42.146036 29478 leader_election.cc:302] T 4b65462f4d1d468392ca83a637347792 P d6568f7cf549467f8a0fde547cd97a6e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ed01a8ae0c3497bbb997a6f77fc46bb, d6568f7cf549467f8a0fde547cd97a6e; no voters:
I20260809 16:20:42.146106 29367 raft_consensus.cc:2454] T 4b65462f4d1d468392ca83a637347792 P 18290c4f39d74d8bb5275a1f9d5f183b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d6568f7cf549467f8a0fde547cd97a6e in term 0.
I20260809 16:20:42.146171 29689 raft_consensus.cc:2781] T 4b65462f4d1d468392ca83a637347792 P d6568f7cf549467f8a0fde547cd97a6e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 16:20:42.146229 29689 raft_consensus.cc:482] T 4b65462f4d1d468392ca83a637347792 P d6568f7cf549467f8a0fde547cd97a6e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 16:20:42.146255 29689 raft_consensus.cc:3037] T 4b65462f4d1d468392ca83a637347792 P d6568f7cf549467f8a0fde547cd97a6e [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:20:42.146975 29689 raft_consensus.cc:504] T 4b65462f4d1d468392ca83a637347792 P d6568f7cf549467f8a0fde547cd97a6e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18290c4f39d74d8bb5275a1f9d5f183b" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 44979 } } peers { permanent_uuid: "1ed01a8ae0c3497bbb997a6f77fc46bb" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 35793 } } peers { permanent_uuid: "d6568f7cf549467f8a0fde547cd97a6e" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 46375 } }
I20260809 16:20:42.147205 29689 leader_election.cc:288] T 4b65462f4d1d468392ca83a637347792 P d6568f7cf549467f8a0fde547cd97a6e [CANDIDATE]: Term 1 election: Requested vote from peers 18290c4f39d74d8bb5275a1f9d5f183b (127.21.51.130:44979), 1ed01a8ae0c3497bbb997a6f77fc46bb (127.21.51.129:35793)
I20260809 16:20:42.147281 29367 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4b65462f4d1d468392ca83a637347792" candidate_uuid: "d6568f7cf549467f8a0fde547cd97a6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "18290c4f39d74d8bb5275a1f9d5f183b"
I20260809 16:20:42.147359 29367 raft_consensus.cc:3037] T 4b65462f4d1d468392ca83a637347792 P 18290c4f39d74d8bb5275a1f9d5f183b [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:20:42.147392 29171 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4b65462f4d1d468392ca83a637347792" candidate_uuid: "d6568f7cf549467f8a0fde547cd97a6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1ed01a8ae0c3497bbb997a6f77fc46bb"
I20260809 16:20:42.147456 29171 raft_consensus.cc:3037] T 4b65462f4d1d468392ca83a637347792 P 1ed01a8ae0c3497bbb997a6f77fc46bb [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:20:42.148010 29367 raft_consensus.cc:2454] T 4b65462f4d1d468392ca83a637347792 P 18290c4f39d74d8bb5275a1f9d5f183b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6568f7cf549467f8a0fde547cd97a6e in term 1.
I20260809 16:20:42.148183 29171 raft_consensus.cc:2454] T 4b65462f4d1d468392ca83a637347792 P 1ed01a8ae0c3497bbb997a6f77fc46bb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6568f7cf549467f8a0fde547cd97a6e in term 1.
I20260809 16:20:42.148203 29475 leader_election.cc:302] T 4b65462f4d1d468392ca83a637347792 P d6568f7cf549467f8a0fde547cd97a6e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 18290c4f39d74d8bb5275a1f9d5f183b, d6568f7cf549467f8a0fde547cd97a6e; no voters:
I20260809 16:20:42.148375 29689 raft_consensus.cc:2781] T 4b65462f4d1d468392ca83a637347792 P d6568f7cf549467f8a0fde547cd97a6e [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:20:42.148517 29689 raft_consensus.cc:686] T 4b65462f4d1d468392ca83a637347792 P d6568f7cf549467f8a0fde547cd97a6e [term 1 LEADER]: Becoming Leader. State: Replica: d6568f7cf549467f8a0fde547cd97a6e, State: Running, Role: LEADER
I20260809 16:20:42.148615 29689 consensus_queue.cc:237] T 4b65462f4d1d468392ca83a637347792 P d6568f7cf549467f8a0fde547cd97a6e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18290c4f39d74d8bb5275a1f9d5f183b" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 44979 } } peers { permanent_uuid: "1ed01a8ae0c3497bbb997a6f77fc46bb" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 35793 } } peers { permanent_uuid: "d6568f7cf549467f8a0fde547cd97a6e" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 46375 } }
I20260809 16:20:42.149454 28949 catalog_manager.cc:5495] T 4b65462f4d1d468392ca83a637347792 P d6568f7cf549467f8a0fde547cd97a6e reported cstate change: term changed from 0 to 1, leader changed from <none> to d6568f7cf549467f8a0fde547cd97a6e (127.21.51.131). New cstate: current_term: 1 leader_uuid: "d6568f7cf549467f8a0fde547cd97a6e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18290c4f39d74d8bb5275a1f9d5f183b" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 44979 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1ed01a8ae0c3497bbb997a6f77fc46bb" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 35793 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d6568f7cf549467f8a0fde547cd97a6e" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 46375 } health_report { overall_health: HEALTHY } } }
I20260809 16:20:42.158015 29674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:42.157151 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33238 (local address 127.21.51.131:46375)
0809 16:20:42.157203 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0809 16:20:42.157205 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:20:42.157297 (+ 92us) server_negotiation.cc:407] Connection header received
0809 16:20:42.157348 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:20:42.157349 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:20:42.157359 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:20:42.157416 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:20:42.157856 (+ 440us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:20:42.157857 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:20:42.157858 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 16:20:42.157861 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:20:42.157876 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 16:20:42.157880 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:20:42.157930 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0809 16:20:42.157962 (+ 32us) server_negotiation.cc:299] Negotiation successful
0809 16:20:42.157972 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260809 16:20:42.159636 29367 raft_consensus.cc:1261] T 4b65462f4d1d468392ca83a637347792 P 18290c4f39d74d8bb5275a1f9d5f183b [term 1 FOLLOWER]: Refusing update from remote peer d6568f7cf549467f8a0fde547cd97a6e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 16:20:42.159641 29171 raft_consensus.cc:1261] T 4b65462f4d1d468392ca83a637347792 P 1ed01a8ae0c3497bbb997a6f77fc46bb [term 1 FOLLOWER]: Refusing update from remote peer d6568f7cf549467f8a0fde547cd97a6e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 16:20:42.159976 29692 consensus_queue.cc:1035] T 4b65462f4d1d468392ca83a637347792 P d6568f7cf549467f8a0fde547cd97a6e [LEADER]: Connected to new peer: Peer: permanent_uuid: "18290c4f39d74d8bb5275a1f9d5f183b" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 44979 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 16:20:42.160109 29689 consensus_queue.cc:1035] T 4b65462f4d1d468392ca83a637347792 P d6568f7cf549467f8a0fde547cd97a6e [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ed01a8ae0c3497bbb997a6f77fc46bb" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 35793 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 16:20:42.162477 29697 mvcc.cc:205] Tried to move back new op lower bound from 7316653843083608064 to 7316653843041665024. Current Snapshot: MvccSnapshot[applied={T|T < 7316653843083608064}]
I20260809 16:20:42.163257 29698 mvcc.cc:205] Tried to move back new op lower bound from 7316653843083608064 to 7316653843041665024. Current Snapshot: MvccSnapshot[applied={T|T < 7316653843083608064}]
I20260809 16:20:42.164896 28949 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:55538:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 16:20:42.164975 28949 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:55538:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 16:20:42.165656 28949 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7c2c6ec5f2fa42a7af8c72505053c32d: Sending DeleteTablet for 3 replicas of tablet 4b65462f4d1d468392ca83a637347792
I20260809 16:20:42.165982 29337 tablet_service.cc:1514] Processing DeleteTablet for tablet 4b65462f4d1d468392ca83a637347792 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:20:42 UTC) from {username='slave'} at 127.0.0.1:52684
I20260809 16:20:42.166038 29146 tablet_service.cc:1514] Processing DeleteTablet for tablet 4b65462f4d1d468392ca83a637347792 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:20:42 UTC) from {username='slave'} at 127.0.0.1:52974
I20260809 16:20:42.166076 29542 tablet_service.cc:1514] Processing DeleteTablet for tablet 4b65462f4d1d468392ca83a637347792 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:20:42 UTC) from {username='slave'} at 127.0.0.1:33224
I20260809 16:20:42.166337 21710 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrNqUCn/build/release/bin/kudu with pid 29050
I20260809 16:20:42.175540 29713 tablet_replica.cc:330] T 4b65462f4d1d468392ca83a637347792 P 18290c4f39d74d8bb5275a1f9d5f183b: stopping tablet replica
I20260809 16:20:42.175693 29713 raft_consensus.cc:2229] T 4b65462f4d1d468392ca83a637347792 P 18290c4f39d74d8bb5275a1f9d5f183b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 16:20:42.175598 29719 tablet_replica.cc:330] T 4b65462f4d1d468392ca83a637347792 P d6568f7cf549467f8a0fde547cd97a6e: stopping tablet replica
I20260809 16:20:42.175824 29713 raft_consensus.cc:2258] T 4b65462f4d1d468392ca83a637347792 P 18290c4f39d74d8bb5275a1f9d5f183b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 16:20:42.175951 29719 raft_consensus.cc:2229] T 4b65462f4d1d468392ca83a637347792 P d6568f7cf549467f8a0fde547cd97a6e [term 1 LEADER]: Raft consensus shutting down.
I20260809 16:20:42.176107 29719 raft_consensus.cc:2258] T 4b65462f4d1d468392ca83a637347792 P d6568f7cf549467f8a0fde547cd97a6e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 16:20:42.176373 29713 ts_tablet_manager.cc:1905] T 4b65462f4d1d468392ca83a637347792 P 18290c4f39d74d8bb5275a1f9d5f183b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 16:20:42.176667 29719 ts_tablet_manager.cc:1905] T 4b65462f4d1d468392ca83a637347792 P d6568f7cf549467f8a0fde547cd97a6e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 16:20:42.178519 29719 ts_tablet_manager.cc:1918] T 4b65462f4d1d468392ca83a637347792 P d6568f7cf549467f8a0fde547cd97a6e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 16:20:42.178678 29719 log.cc:1196] T 4b65462f4d1d468392ca83a637347792 P d6568f7cf549467f8a0fde547cd97a6e: Deleting WAL directory at /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786292417722232-21710-0/minicluster-data/ts-2/wal/wals/4b65462f4d1d468392ca83a637347792
I20260809 16:20:42.179091 29719 ts_tablet_manager.cc:1939] T 4b65462f4d1d468392ca83a637347792 P d6568f7cf549467f8a0fde547cd97a6e: Deleting consensus metadata
I20260809 16:20:42.179250 29713 ts_tablet_manager.cc:1918] T 4b65462f4d1d468392ca83a637347792 P 18290c4f39d74d8bb5275a1f9d5f183b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 16:20:42.179330 29713 log.cc:1196] T 4b65462f4d1d468392ca83a637347792 P 18290c4f39d74d8bb5275a1f9d5f183b: Deleting WAL directory at /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786292417722232-21710-0/minicluster-data/ts-1/wal/wals/4b65462f4d1d468392ca83a637347792
I20260809 16:20:42.179601 29713 ts_tablet_manager.cc:1939] T 4b65462f4d1d468392ca83a637347792 P 18290c4f39d74d8bb5275a1f9d5f183b: Deleting consensus metadata
I20260809 16:20:42.179802 28917 catalog_manager.cc:4841] TS d6568f7cf549467f8a0fde547cd97a6e (127.21.51.131:46375): tablet 4b65462f4d1d468392ca83a637347792 (table test-table [id=566b23ccff5d417b98844e4ad50cc11b]) successfully deleted
I20260809 16:20:42.180094 28915 catalog_manager.cc:4841] TS 18290c4f39d74d8bb5275a1f9d5f183b (127.21.51.130:44979): tablet 4b65462f4d1d468392ca83a637347792 (table test-table [id=566b23ccff5d417b98844e4ad50cc11b]) successfully deleted
W20260809 16:20:42.183944 28917 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.21.51.129:35793 (error 108)
I20260809 16:20:42.184146 21710 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrNqUCn/build/release/bin/kudu with pid 29250
I20260809 16:20:42.190447 29669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:42.188922 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.129:35793 (local address 127.0.0.1:52980)
0809 16:20:42.190334 (+ 1412us) negotiation.cc:107] Waiting for socket to connect
0809 16:20:42.190402 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.51.129:35793: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1382}
W20260809 16:20:42.191236 28917 catalog_manager.cc:4584] TS 1ed01a8ae0c3497bbb997a6f77fc46bb (127.21.51.129:35793): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 4b65462f4d1d468392ca83a637347792: Network error: Client connection negotiation failed: client connection to 127.21.51.129:35793: connect: Connection refused (error 111)
I20260809 16:20:42.197109 21710 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrNqUCn/build/release/bin/kudu with pid 29455
I20260809 16:20:42.204818 21710 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrNqUCn/build/release/bin/kudu with pid 28890
2026-08-09T16:20:42Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (645 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260809 16:20:42.227741 21710 mini_oidc.cc:152] Starting JWKS server
I20260809 16:20:42.228915 21710 webserver.cc:460] Webserver started at https://127.0.0.1:41427/ using document root <none> and password file <none>
I20260809 16:20:42.229036 21710 mini_oidc.cc:189] Starting OIDC Discovery server
I20260809 16:20:42.229223 21710 webserver.cc:460] Webserver started at http://127.0.0.1:44619/ using document root <none> and password file <none>
2026-08-09T16:20:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T16:20:42Z Disabled control of system clock
I20260809 16:20:42.239369 21710 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrNqUCn/build/release/bin/kudu
/tmp/dist-test-taskrNqUCn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786292417722232-21710-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786292417722232-21710-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786292417722232-21710-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786292417722232-21710-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.51.190:39321
--webserver_interface=127.21.51.190
--webserver_port=0
--builtin_ntp_servers=127.21.51.148:43177
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.51.190:39321
--enable_jwt_token_auth=true
--jwks_url=https://localhost:41427/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786292417722232-21710-0/testchainca.cert with env {}
W20260809 16:20:42.323732 29729 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:20:42.323918 29729 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:20:42.323939 29729 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:20:42.325124 29729 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 16:20:42.325163 29729 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:20:42.325176 29729 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:20:42.325189 29729 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 16:20:42.325204 29729 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260809 16:20:42.325255 29729 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260809 16:20:42.325273 29729 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:41427/jwks.json
I20260809 16:20:42.326857 29729 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.51.148:43177
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786292417722232-21710-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786292417722232-21710-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.51.190:39321
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.51.190:39321
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:41427/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786292417722232-21710-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.51.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786292417722232-21710-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29729
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786292417722232-21710-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:20:42.327096 29729 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:20:42.327314 29729 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:20:42.332201 29745 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:20:42.332214 29742 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:20:42.332216 29743 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:20:42.332777 29729 server_base.cc:1034] running on GCE node
I20260809 16:20:42.333000 29729 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:20:42.333230 29729 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:20:42.334372 29729 hybrid_clock.cc:648] HybridClock initialized: now 1786292442334353 us; error 44 us; skew 500 ppm
I20260809 16:20:42.335604 29729 webserver.cc:460] Webserver started at http://127.21.51.190:37829/ using document root <none> and password file <none>
I20260809 16:20:42.335920 29729 fs_manager.cc:359] Metadata directory not provided
I20260809 16:20:42.335973 29729 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:20:42.336143 29729 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:20:42.337255 29729 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786292417722232-21710-0/minicluster-data/master-0/data/instance:
uuid: "9e20415043e349ebbbf347bdb7fb6b33"
format_stamp: "Formatted at 2026-08-09 16:20:42 on dist-test-slave-s11t"
server_key: "94e77c441cdc621e68bf1ada61d4aa47"
server_key_iv: "0fce24ed8d9b80fd09f549d02bf503c4"
server_key_version: "encryptionkey@0"
I20260809 16:20:42.337616 29729 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786292417722232-21710-0/minicluster-data/master-0/wal/instance:
uuid: "9e20415043e349ebbbf347bdb7fb6b33"
format_stamp: "Formatted at 2026-08-09 16:20:42 on dist-test-slave-s11t"
server_key: "94e77c441cdc621e68bf1ada61d4aa47"
server_key_iv: "0fce24ed8d9b80fd09f549d02bf503c4"
server_key_version: "encryptionkey@0"
I20260809 16:20:42.338982 29729 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 16:20:42.339777 29753 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:20:42.339972 29729 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:20:42.340098 29729 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786292417722232-21710-0/minicluster-data/master-0/data,/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786292417722232-21710-0/minicluster-data/master-0/wal
uuid: "9e20415043e349ebbbf347bdb7fb6b33"
format_stamp: "Formatted at 2026-08-09 16:20:42 on dist-test-slave-s11t"
server_key: "94e77c441cdc621e68bf1ada61d4aa47"
server_key_iv: "0fce24ed8d9b80fd09f549d02bf503c4"
server_key_version: "encryptionkey@0"
I20260809 16:20:42.340202 29729 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786292417722232-21710-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786292417722232-21710-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786292417722232-21710-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:20:42.351543 29729 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:20:42.351845 29729 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:20:42.351994 29729 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:20:42.356145 29729 rpc_server.cc:307] RPC server started. Bound to: 127.21.51.190:39321
I20260809 16:20:42.356197 29830 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.51.190:39321 every 8 connection(s)
I20260809 16:20:42.356537 29729 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786292417722232-21710-0/minicluster-data/master-0/data/info.pb
I20260809 16:20:42.357599 29834 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:20:42.359763 29834 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9e20415043e349ebbbf347bdb7fb6b33: Bootstrap starting.
I20260809 16:20:42.360284 29834 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9e20415043e349ebbbf347bdb7fb6b33: Neither blocks nor log segments found. Creating new log.
I20260809 16:20:42.360514 29834 log.cc:824] T 00000000000000000000000000000000 P 9e20415043e349ebbbf347bdb7fb6b33: Log is configured to *not* fsync() on all Append() calls
I20260809 16:20:42.361205 29834 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9e20415043e349ebbbf347bdb7fb6b33: No bootstrap required, opened a new log
I20260809 16:20:42.362677 29834 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9e20415043e349ebbbf347bdb7fb6b33 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e20415043e349ebbbf347bdb7fb6b33" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 39321 } }
I20260809 16:20:42.362834 29834 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9e20415043e349ebbbf347bdb7fb6b33 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:20:42.362864 29834 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9e20415043e349ebbbf347bdb7fb6b33 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e20415043e349ebbbf347bdb7fb6b33, State: Initialized, Role: FOLLOWER
I20260809 16:20:42.362994 29834 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9e20415043e349ebbbf347bdb7fb6b33 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e20415043e349ebbbf347bdb7fb6b33" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 39321 } }
I20260809 16:20:42.363085 29834 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9e20415043e349ebbbf347bdb7fb6b33 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 16:20:42.363109 29834 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9e20415043e349ebbbf347bdb7fb6b33 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 16:20:42.363137 29834 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9e20415043e349ebbbf347bdb7fb6b33 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:20:42.363593 29834 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9e20415043e349ebbbf347bdb7fb6b33 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e20415043e349ebbbf347bdb7fb6b33" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 39321 } }
I20260809 16:20:42.363673 29834 leader_election.cc:302] T 00000000000000000000000000000000 P 9e20415043e349ebbbf347bdb7fb6b33 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9e20415043e349ebbbf347bdb7fb6b33; no voters:
I20260809 16:20:42.363852 29834 leader_election.cc:288] T 00000000000000000000000000000000 P 9e20415043e349ebbbf347bdb7fb6b33 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 16:20:42.363921 29837 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9e20415043e349ebbbf347bdb7fb6b33 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:20:42.363955 21710 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrNqUCn/build/release/bin/kudu as pid 29729
I20260809 16:20:42.364051 21710 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786292417722232-21710-0/minicluster-data/master-0/wal/instance
I20260809 16:20:42.364076 29837 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9e20415043e349ebbbf347bdb7fb6b33 [term 1 LEADER]: Becoming Leader. State: Replica: 9e20415043e349ebbbf347bdb7fb6b33, State: Running, Role: LEADER
I20260809 16:20:42.364136 29834 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9e20415043e349ebbbf347bdb7fb6b33 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 16:20:42.364182 29837 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9e20415043e349ebbbf347bdb7fb6b33 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e20415043e349ebbbf347bdb7fb6b33" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 39321 } }
I20260809 16:20:42.364276 21710 external_mini_cluster.cc:1428] Setting key becd566e36f64834429530f04bfe806d
I20260809 16:20:42.364672 29838 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9e20415043e349ebbbf347bdb7fb6b33 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9e20415043e349ebbbf347bdb7fb6b33. Latest consensus state: current_term: 1 leader_uuid: "9e20415043e349ebbbf347bdb7fb6b33" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e20415043e349ebbbf347bdb7fb6b33" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 39321 } } }
I20260809 16:20:42.364809 29838 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9e20415043e349ebbbf347bdb7fb6b33 [sys.catalog]: This master's current role is: LEADER
I20260809 16:20:42.365023 29851 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 16:20:42.365497 29840 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9e20415043e349ebbbf347bdb7fb6b33 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9e20415043e349ebbbf347bdb7fb6b33" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e20415043e349ebbbf347bdb7fb6b33" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 39321 } } }
I20260809 16:20:42.365558 29851 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 16:20:42.365680 29840 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9e20415043e349ebbbf347bdb7fb6b33 [sys.catalog]: This master's current role is: LEADER
I20260809 16:20:42.367267 29851 catalog_manager.cc:1347] Generated new cluster ID: f143665b890a4540aea3063739ac313a
I20260809 16:20:42.367323 29851 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 16:20:42.371388 29854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:42.365059 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44880 (local address 127.21.51.190:39321)
0809 16:20:42.365324 (+ 265us) server_negotiation.cc:206] Beginning negotiation
0809 16:20:42.365333 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 16:20:42.365623 (+ 290us) server_negotiation.cc:407] Connection header received
0809 16:20:42.365846 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:20:42.365853 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:20:42.365954 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:20:42.366088 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:20:42.366442 (+ 354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:42.367793 (+ 1351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:42.368899 (+ 1106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:42.369530 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:42.369628 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:42.370536 (+ 908us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:20:42.370539 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:20:42.370540 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 16:20:42.370740 (+ 200us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:20:42.370790 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 16:20:42.370803 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:20:42.370854 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0809 16:20:42.371107 (+ 253us) server_negotiation.cc:299] Negotiation successful
0809 16:20:42.371182 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":177,"thread_start_us":77,"threads_started":1}
I20260809 16:20:42.372262 21710 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 16:20:42.377141 29851 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 16:20:42.377717 29851 catalog_manager.cc:1504] Loading token signing keys...
I20260809 16:20:42.379969 29851 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9e20415043e349ebbbf347bdb7fb6b33: Generated new TSK 0
I20260809 16:20:42.380138 29851 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260809 16:20:42.401818 29883 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260809 16:20:42.402070 29883 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
W20260809 16:20:42.402586 29882 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:20:42.386486 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:39321 (local address 127.0.0.1:44896)
0809 16:20:42.386693 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0809 16:20:42.386702 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 16:20:42.386791 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:20:42.386965 (+ 174us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:20:42.386966 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:20:42.386968 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0809 16:20:42.387029 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:20:42.387031 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:20:42.387684 (+ 653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:20:42.387685 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:20:42.388269 (+ 584us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:20:42.388271 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:20:42.388664 (+ 393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:20:42.388665 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:20:42.388747 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:42.388781 (+ 34us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0809 16:20:42.402213 (+ 13432us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41427/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0809 16:20:42.402304 (+ 91us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.21.51.190:39321: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41427/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":101,"threads_started":1}
I20260809 16:20:42.402894 29854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:42.386549 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44896 (local address 127.21.51.190:39321)
0809 16:20:42.386613 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0809 16:20:42.386615 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:20:42.386769 (+ 154us) server_negotiation.cc:407] Connection header received
0809 16:20:42.386881 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:20:42.386882 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:20:42.386894 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:20:42.386958 (+ 64us) server_negotiation.cc:226] Negotiated authn=JWT
0809 16:20:42.387106 (+ 148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:42.387576 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:42.388366 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:42.388565 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:42.388635 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:42.388892 (+ 257us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0809 16:20:42.402066 (+ 13174us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41427/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0809 16:20:42.402825 (+ 759us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:44896: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41427/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":41}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.21.51.190:39321: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41427/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
W20260809 16:20:42.403057 29854 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:44896: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41427/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
I20260809 16:20:42.403069 21710 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrNqUCn/build/release/bin/kudu with pid 29729
2026-08-09T16:20:42Z chronyd exiting
I20260809 16:20:42.447851 21710 test_util.cc:175] -----------------------------------------------
I20260809 16:20:42.447934 21710 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786292417722232-21710-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (221 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260809 16:20:42.448750 21710 mini_oidc.cc:152] Starting JWKS server
I20260809 16:20:42.449455 21710 webserver.cc:460] Webserver started at https://127.0.0.1:46725/ using document root <none> and password file <none>
I20260809 16:20:42.449497 21710 mini_oidc.cc:189] Starting OIDC Discovery server
I20260809 16:20:42.449635 21710 webserver.cc:460] Webserver started at http://127.0.0.1:41949/ using document root <none> and password file <none>
2026-08-09T16:20:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T16:20:42Z Disabled control of system clock
I20260809 16:20:42.458590 21710 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrNqUCn/build/release/bin/kudu
/tmp/dist-test-taskrNqUCn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786292417722232-21710-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786292417722232-21710-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786292417722232-21710-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786292417722232-21710-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.51.190:44821
--webserver_interface=127.21.51.190
--webserver_port=0
--builtin_ntp_servers=127.21.51.148:35135
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.51.190:44821
--enable_jwt_token_auth=true
--jwks_url=https://localhost:46725/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786292417722232-21710-0/testchainca.cert with env {}
W20260809 16:20:42.540084 29890 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:20:42.540290 29890 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:20:42.540313 29890 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:20:42.541417 29890 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 16:20:42.541456 29890 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:20:42.541469 29890 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:20:42.541481 29890 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 16:20:42.541493 29890 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260809 16:20:42.541505 29890 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260809 16:20:42.541517 29890 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:46725/jwks.json
I20260809 16:20:42.542991 29890 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.51.148:35135
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786292417722232-21710-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786292417722232-21710-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.51.190:44821
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.51.190:44821
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:46725/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786292417722232-21710-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.51.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786292417722232-21710-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29890
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786292417722232-21710-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:20:42.543198 29890 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:20:42.543370 29890 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:20:42.548260 29897 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:20:42.548249 29900 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:20:42.548264 29898 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:20:42.548550 29890 server_base.cc:1034] running on GCE node
I20260809 16:20:42.549722 29890 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:20:42.550024 29890 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:20:42.551198 29890 hybrid_clock.cc:648] HybridClock initialized: now 1786292442551176 us; error 38 us; skew 500 ppm
I20260809 16:20:42.552429 29890 webserver.cc:460] Webserver started at http://127.21.51.190:38077/ using document root <none> and password file <none>
I20260809 16:20:42.552675 29890 fs_manager.cc:359] Metadata directory not provided
I20260809 16:20:42.552809 29890 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:20:42.552995 29890 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:20:42.554047 29890 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786292417722232-21710-0/minicluster-data/master-0/data/instance:
uuid: "7570748592ef478eac1c66b36380071d"
format_stamp: "Formatted at 2026-08-09 16:20:42 on dist-test-slave-s11t"
server_key: "c2e8a7273f0a61a63241812d2be937d8"
server_key_iv: "ea26c3e70ab3c1bde65356494306008b"
server_key_version: "encryptionkey@0"
I20260809 16:20:42.554428 29890 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786292417722232-21710-0/minicluster-data/master-0/wal/instance:
uuid: "7570748592ef478eac1c66b36380071d"
format_stamp: "Formatted at 2026-08-09 16:20:42 on dist-test-slave-s11t"
server_key: "c2e8a7273f0a61a63241812d2be937d8"
server_key_iv: "ea26c3e70ab3c1bde65356494306008b"
server_key_version: "encryptionkey@0"
I20260809 16:20:42.555738 29890 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 16:20:42.556480 29906 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:20:42.556658 29890 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 16:20:42.556799 29890 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786292417722232-21710-0/minicluster-data/master-0/data,/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786292417722232-21710-0/minicluster-data/master-0/wal
uuid: "7570748592ef478eac1c66b36380071d"
format_stamp: "Formatted at 2026-08-09 16:20:42 on dist-test-slave-s11t"
server_key: "c2e8a7273f0a61a63241812d2be937d8"
server_key_iv: "ea26c3e70ab3c1bde65356494306008b"
server_key_version: "encryptionkey@0"
I20260809 16:20:42.556910 29890 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786292417722232-21710-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786292417722232-21710-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786292417722232-21710-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:20:42.567101 29890 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:20:42.567413 29890 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:20:42.567569 29890 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:20:42.571681 29890 rpc_server.cc:307] RPC server started. Bound to: 127.21.51.190:44821
I20260809 16:20:42.571725 29979 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.51.190:44821 every 8 connection(s)
I20260809 16:20:42.572095 29890 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786292417722232-21710-0/minicluster-data/master-0/data/info.pb
I20260809 16:20:42.572819 29980 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:20:42.572988 21710 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrNqUCn/build/release/bin/kudu as pid 29890
I20260809 16:20:42.573105 21710 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786292417722232-21710-0/minicluster-data/master-0/wal/instance
I20260809 16:20:42.573312 21710 external_mini_cluster.cc:1428] Setting key e8c28d0d15204b8c186bab0701c31df2
I20260809 16:20:42.577443 29980 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7570748592ef478eac1c66b36380071d: Bootstrap starting.
I20260809 16:20:42.578195 29980 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7570748592ef478eac1c66b36380071d: Neither blocks nor log segments found. Creating new log.
I20260809 16:20:42.578536 29980 log.cc:824] T 00000000000000000000000000000000 P 7570748592ef478eac1c66b36380071d: Log is configured to *not* fsync() on all Append() calls
I20260809 16:20:42.579474 29980 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7570748592ef478eac1c66b36380071d: No bootstrap required, opened a new log
I20260809 16:20:42.580276 29983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:42.573916 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34580 (local address 127.21.51.190:44821)
0809 16:20:42.574187 (+ 271us) server_negotiation.cc:206] Beginning negotiation
0809 16:20:42.574197 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 16:20:42.574233 (+ 36us) server_negotiation.cc:407] Connection header received
0809 16:20:42.574934 (+ 701us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:20:42.574944 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:20:42.575036 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:20:42.575192 (+ 156us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:20:42.575369 (+ 177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:42.576912 (+ 1543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:42.577829 (+ 917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:42.578523 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:42.578666 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:42.579498 (+ 832us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:20:42.579503 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:20:42.579503 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 16:20:42.579748 (+ 245us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:20:42.579793 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 16:20:42.579808 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:20:42.579883 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0809 16:20:42.579961 (+ 78us) server_negotiation.cc:299] Negotiation successful
0809 16:20:42.580047 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":159,"thread_start_us":87,"threads_started":1}
I20260809 16:20:42.581427 21710 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 16:20:42.582165 29980 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7570748592ef478eac1c66b36380071d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7570748592ef478eac1c66b36380071d" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 44821 } }
I20260809 16:20:42.582356 29980 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7570748592ef478eac1c66b36380071d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:20:42.582432 29980 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7570748592ef478eac1c66b36380071d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7570748592ef478eac1c66b36380071d, State: Initialized, Role: FOLLOWER
I20260809 16:20:42.582558 29980 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7570748592ef478eac1c66b36380071d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7570748592ef478eac1c66b36380071d" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 44821 } }
I20260809 16:20:42.582648 29980 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7570748592ef478eac1c66b36380071d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 16:20:42.582701 29980 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7570748592ef478eac1c66b36380071d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 16:20:42.582759 29980 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7570748592ef478eac1c66b36380071d [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:20:42.583333 29980 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7570748592ef478eac1c66b36380071d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7570748592ef478eac1c66b36380071d" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 44821 } }
I20260809 16:20:42.583463 29980 leader_election.cc:302] T 00000000000000000000000000000000 P 7570748592ef478eac1c66b36380071d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7570748592ef478eac1c66b36380071d; no voters:
I20260809 16:20:42.583639 29980 leader_election.cc:288] T 00000000000000000000000000000000 P 7570748592ef478eac1c66b36380071d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 16:20:42.583690 29985 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7570748592ef478eac1c66b36380071d [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:20:42.583906 29985 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7570748592ef478eac1c66b36380071d [term 1 LEADER]: Becoming Leader. State: Replica: 7570748592ef478eac1c66b36380071d, State: Running, Role: LEADER
I20260809 16:20:42.583937 29980 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7570748592ef478eac1c66b36380071d [sys.catalog]: configured and running, proceeding with master startup.
I20260809 16:20:42.583997 29985 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7570748592ef478eac1c66b36380071d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7570748592ef478eac1c66b36380071d" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 44821 } }
I20260809 16:20:42.584357 29986 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7570748592ef478eac1c66b36380071d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7570748592ef478eac1c66b36380071d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7570748592ef478eac1c66b36380071d" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 44821 } } }
I20260809 16:20:42.584381 29987 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7570748592ef478eac1c66b36380071d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7570748592ef478eac1c66b36380071d. Latest consensus state: current_term: 1 leader_uuid: "7570748592ef478eac1c66b36380071d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7570748592ef478eac1c66b36380071d" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 44821 } } }
I20260809 16:20:42.584476 29986 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7570748592ef478eac1c66b36380071d [sys.catalog]: This master's current role is: LEADER
I20260809 16:20:42.584499 29987 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7570748592ef478eac1c66b36380071d [sys.catalog]: This master's current role is: LEADER
I20260809 16:20:42.584811 29995 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 16:20:42.585369 29995 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 16:20:42.586907 29995 catalog_manager.cc:1347] Generated new cluster ID: c8760c3e651346d780be5b7565615c08
I20260809 16:20:42.586964 29995 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 16:20:42.592509 29995 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 16:20:42.593048 29995 catalog_manager.cc:1504] Loading token signing keys...
I20260809 16:20:42.599851 29995 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7570748592ef478eac1c66b36380071d: Generated new TSK 0
I20260809 16:20:42.600051 29995 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260809 16:20:42.626214 30022 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260809 16:20:42.626312 30022 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260809 16:20:42.626646 29983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:42.611932 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34594 (local address 127.21.51.190:44821)
0809 16:20:42.611997 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0809 16:20:42.612000 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 16:20:42.612105 (+ 105us) server_negotiation.cc:407] Connection header received
0809 16:20:42.612184 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:20:42.612184 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:20:42.612195 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:20:42.612254 (+ 59us) server_negotiation.cc:226] Negotiated authn=JWT
0809 16:20:42.612428 (+ 174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:42.612989 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:42.614209 (+ 1220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:42.614403 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:42.614451 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:42.614648 (+ 197us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0809 16:20:42.626459 (+ 11811us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46725/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0809 16:20:42.626592 (+ 133us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:34594: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46725/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":42}
W20260809 16:20:42.626724 29983 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:34594: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46725/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260809 16:20:42.626855 30020 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:20:42.611814 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:44821 (local address 127.0.0.1:34594)
0809 16:20:42.612028 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0809 16:20:42.612040 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 16:20:42.612112 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:20:42.612281 (+ 169us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:20:42.612283 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:20:42.612284 (+ 1us) client_negotiation.cc:189] Negotiated authn=JWT
0809 16:20:42.612359 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:20:42.612362 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:20:42.613067 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:20:42.613068 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:20:42.614124 (+ 1056us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:20:42.614128 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:20:42.614481 (+ 353us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:20:42.614483 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:20:42.614552 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:42.614559 (+ 7us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0809 16:20:42.626588 (+ 12029us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46725/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0809 16:20:42.626751 (+ 163us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.21.51.190:44821: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46725/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":139,"threads_started":1}
I20260809 16:20:42.627283 21710 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrNqUCn/build/release/bin/kudu with pid 29890
2026-08-09T16:20:42Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (227 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260809 16:20:42.676168 21710 mini_oidc.cc:152] Starting JWKS server
I20260809 16:20:42.676983 21710 webserver.cc:460] Webserver started at https://127.0.0.1:33535/ using document root <none> and password file <none>
I20260809 16:20:42.677035 21710 mini_oidc.cc:189] Starting OIDC Discovery server
I20260809 16:20:42.677158 21710 webserver.cc:460] Webserver started at http://127.0.0.1:43829/ using document root <none> and password file <none>
2026-08-09T16:20:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T16:20:42Z Disabled control of system clock
I20260809 16:20:42.686671 21710 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrNqUCn/build/release/bin/kudu
/tmp/dist-test-taskrNqUCn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786292417722232-21710-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786292417722232-21710-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786292417722232-21710-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786292417722232-21710-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.51.190:38831
--webserver_interface=127.21.51.190
--webserver_port=0
--builtin_ntp_servers=127.21.51.148:35127
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.51.190:38831
--enable_jwt_token_auth=true
--jwks_url=https://localhost:33535/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260809 16:20:42.783370 30036 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:20:42.783576 30036 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:20:42.783600 30036 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:20:42.784906 30036 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 16:20:42.784950 30036 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:20:42.784965 30036 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:20:42.784978 30036 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 16:20:42.784992 30036 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260809 16:20:42.785005 30036 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260809 16:20:42.785018 30036 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:33535/jwks.json
I20260809 16:20:42.786468 30036 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.51.148:35127
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786292417722232-21710-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786292417722232-21710-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.51.190:38831
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.51.190:38831
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:33535/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786292417722232-21710-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.51.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30036
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786292417722232-21710-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:20:42.786670 30036 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:20:42.786926 30036 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:20:42.792258 30047 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:20:42.792258 30042 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:20:42.792305 30043 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:20:42.792620 30036 server_base.cc:1034] running on GCE node
I20260809 16:20:42.792920 30036 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:20:42.793215 30036 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:20:42.794377 30036 hybrid_clock.cc:648] HybridClock initialized: now 1786292442794355 us; error 38 us; skew 500 ppm
I20260809 16:20:42.795636 30036 webserver.cc:460] Webserver started at http://127.21.51.190:43091/ using document root <none> and password file <none>
I20260809 16:20:42.795951 30036 fs_manager.cc:359] Metadata directory not provided
I20260809 16:20:42.796028 30036 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:20:42.796181 30036 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:20:42.797302 30036 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786292417722232-21710-0/minicluster-data/master-0/data/instance:
uuid: "c485d9fe1eb743f68ff30a8abe157912"
format_stamp: "Formatted at 2026-08-09 16:20:42 on dist-test-slave-s11t"
server_key: "81604751636eb56f3d7ca5475b14641d"
server_key_iv: "0aa2fa22fd382764354251775065d2f3"
server_key_version: "encryptionkey@0"
I20260809 16:20:42.797670 30036 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786292417722232-21710-0/minicluster-data/master-0/wal/instance:
uuid: "c485d9fe1eb743f68ff30a8abe157912"
format_stamp: "Formatted at 2026-08-09 16:20:42 on dist-test-slave-s11t"
server_key: "81604751636eb56f3d7ca5475b14641d"
server_key_iv: "0aa2fa22fd382764354251775065d2f3"
server_key_version: "encryptionkey@0"
I20260809 16:20:42.799010 30036 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 16:20:42.799824 30055 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:20:42.800028 30036 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:20:42.800137 30036 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786292417722232-21710-0/minicluster-data/master-0/data,/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786292417722232-21710-0/minicluster-data/master-0/wal
uuid: "c485d9fe1eb743f68ff30a8abe157912"
format_stamp: "Formatted at 2026-08-09 16:20:42 on dist-test-slave-s11t"
server_key: "81604751636eb56f3d7ca5475b14641d"
server_key_iv: "0aa2fa22fd382764354251775065d2f3"
server_key_version: "encryptionkey@0"
I20260809 16:20:42.800213 30036 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786292417722232-21710-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786292417722232-21710-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786292417722232-21710-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:20:42.818312 30036 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:20:42.818615 30036 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:20:42.818775 30036 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:20:42.823086 30036 rpc_server.cc:307] RPC server started. Bound to: 127.21.51.190:38831
I20260809 16:20:42.823122 30133 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.51.190:38831 every 8 connection(s)
I20260809 16:20:42.823452 30036 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786292417722232-21710-0/minicluster-data/master-0/data/info.pb
I20260809 16:20:42.824126 30135 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:20:42.826885 30135 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c485d9fe1eb743f68ff30a8abe157912: Bootstrap starting.
I20260809 16:20:42.827461 30135 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c485d9fe1eb743f68ff30a8abe157912: Neither blocks nor log segments found. Creating new log.
I20260809 16:20:42.827750 30135 log.cc:824] T 00000000000000000000000000000000 P c485d9fe1eb743f68ff30a8abe157912: Log is configured to *not* fsync() on all Append() calls
I20260809 16:20:42.828449 30135 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c485d9fe1eb743f68ff30a8abe157912: No bootstrap required, opened a new log
I20260809 16:20:42.830078 30135 raft_consensus.cc:348] T 00000000000000000000000000000000 P c485d9fe1eb743f68ff30a8abe157912 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c485d9fe1eb743f68ff30a8abe157912" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 38831 } }
I20260809 16:20:42.830224 30135 raft_consensus.cc:374] T 00000000000000000000000000000000 P c485d9fe1eb743f68ff30a8abe157912 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:20:42.830250 30135 raft_consensus.cc:729] T 00000000000000000000000000000000 P c485d9fe1eb743f68ff30a8abe157912 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c485d9fe1eb743f68ff30a8abe157912, State: Initialized, Role: FOLLOWER
I20260809 16:20:42.830394 30135 consensus_queue.cc:260] T 00000000000000000000000000000000 P c485d9fe1eb743f68ff30a8abe157912 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c485d9fe1eb743f68ff30a8abe157912" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 38831 } }
I20260809 16:20:42.830453 30135 raft_consensus.cc:388] T 00000000000000000000000000000000 P c485d9fe1eb743f68ff30a8abe157912 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 16:20:42.830505 30135 raft_consensus.cc:482] T 00000000000000000000000000000000 P c485d9fe1eb743f68ff30a8abe157912 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 16:20:42.830557 30135 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c485d9fe1eb743f68ff30a8abe157912 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:20:42.831103 30135 raft_consensus.cc:504] T 00000000000000000000000000000000 P c485d9fe1eb743f68ff30a8abe157912 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c485d9fe1eb743f68ff30a8abe157912" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 38831 } }
I20260809 16:20:42.831219 30135 leader_election.cc:302] T 00000000000000000000000000000000 P c485d9fe1eb743f68ff30a8abe157912 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c485d9fe1eb743f68ff30a8abe157912; no voters:
I20260809 16:20:42.831395 30135 leader_election.cc:288] T 00000000000000000000000000000000 P c485d9fe1eb743f68ff30a8abe157912 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 16:20:42.831423 30142 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c485d9fe1eb743f68ff30a8abe157912 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:20:42.831544 30142 raft_consensus.cc:686] T 00000000000000000000000000000000 P c485d9fe1eb743f68ff30a8abe157912 [term 1 LEADER]: Becoming Leader. State: Replica: c485d9fe1eb743f68ff30a8abe157912, State: Running, Role: LEADER
I20260809 16:20:42.831552 21710 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrNqUCn/build/release/bin/kudu as pid 30036
I20260809 16:20:42.831672 30135 sys_catalog.cc:564] T 00000000000000000000000000000000 P c485d9fe1eb743f68ff30a8abe157912 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 16:20:42.831677 21710 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786292417722232-21710-0/minicluster-data/master-0/wal/instance
I20260809 16:20:42.831637 30142 consensus_queue.cc:237] T 00000000000000000000000000000000 P c485d9fe1eb743f68ff30a8abe157912 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c485d9fe1eb743f68ff30a8abe157912" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 38831 } }
I20260809 16:20:42.831890 21710 external_mini_cluster.cc:1428] Setting key ab4a6d7b49449f4517568f6d713e4e37
I20260809 16:20:42.832036 30144 sys_catalog.cc:455] T 00000000000000000000000000000000 P c485d9fe1eb743f68ff30a8abe157912 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c485d9fe1eb743f68ff30a8abe157912" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c485d9fe1eb743f68ff30a8abe157912" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 38831 } } }
I20260809 16:20:42.832479 30144 sys_catalog.cc:458] T 00000000000000000000000000000000 P c485d9fe1eb743f68ff30a8abe157912 [sys.catalog]: This master's current role is: LEADER
I20260809 16:20:42.832386 30145 sys_catalog.cc:455] T 00000000000000000000000000000000 P c485d9fe1eb743f68ff30a8abe157912 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c485d9fe1eb743f68ff30a8abe157912. Latest consensus state: current_term: 1 leader_uuid: "c485d9fe1eb743f68ff30a8abe157912" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c485d9fe1eb743f68ff30a8abe157912" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 38831 } } }
I20260809 16:20:42.832762 30145 sys_catalog.cc:458] T 00000000000000000000000000000000 P c485d9fe1eb743f68ff30a8abe157912 [sys.catalog]: This master's current role is: LEADER
I20260809 16:20:42.832866 30159 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 16:20:42.833384 30159 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 16:20:42.835005 30159 catalog_manager.cc:1347] Generated new cluster ID: 3fa5a7e3d6ac4abfa46a19bab18c8bcd
I20260809 16:20:42.835063 30159 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 16:20:42.838063 30160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:42.832553 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46614 (local address 127.21.51.190:38831)
0809 16:20:42.832855 (+ 302us) server_negotiation.cc:206] Beginning negotiation
0809 16:20:42.832864 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 16:20:42.833147 (+ 283us) server_negotiation.cc:407] Connection header received
0809 16:20:42.833277 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:20:42.833285 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:20:42.833359 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:20:42.833504 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:20:42.833956 (+ 452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:42.835274 (+ 1318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:42.836052 (+ 778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:42.836483 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:42.836621 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:42.837407 (+ 786us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:20:42.837411 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:20:42.837411 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 16:20:42.837592 (+ 181us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:20:42.837625 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 16:20:42.837637 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:20:42.837732 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0809 16:20:42.837807 (+ 75us) server_negotiation.cc:299] Negotiation successful
0809 16:20:42.837885 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":215,"thread_start_us":92,"threads_started":1}
I20260809 16:20:42.839102 21710 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 16:20:42.843685 30159 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 16:20:42.844149 30159 catalog_manager.cc:1504] Loading token signing keys...
I20260809 16:20:42.854058 30159 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c485d9fe1eb743f68ff30a8abe157912: Generated new TSK 0
I20260809 16:20:42.854242 30159 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 16:20:42.867931 30160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:42.863789 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46618 (local address 127.21.51.190:38831)
0809 16:20:42.863834 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0809 16:20:42.863836 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:20:42.863928 (+ 92us) server_negotiation.cc:407] Connection header received
0809 16:20:42.864000 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:20:42.864000 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:20:42.864010 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:20:42.864070 (+ 60us) server_negotiation.cc:226] Negotiated authn=JWT
0809 16:20:42.864263 (+ 193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:42.864984 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:42.866139 (+ 1155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:42.866372 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:42.866439 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:42.866657 (+ 218us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0809 16:20:42.867687 (+ 1030us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33535/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0809 16:20:42.867834 (+ 147us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:46618: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33535/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":24}
W20260809 16:20:42.867995 30160 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:46618: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33535/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260809 16:20:42.868271 30180 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:20:42.863689 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:38831 (local address 127.0.0.1:46618)
0809 16:20:42.863851 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0809 16:20:42.863862 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 16:20:42.863931 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:20:42.864087 (+ 156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:20:42.864088 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:20:42.864090 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0809 16:20:42.864159 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:20:42.864162 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:20:42.865074 (+ 912us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:20:42.865075 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:20:42.866040 (+ 965us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:20:42.866043 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:20:42.866461 (+ 418us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:20:42.866462 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:20:42.866530 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:42.866536 (+ 6us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0809 16:20:42.867804 (+ 1268us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33535/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0809 16:20:42.867909 (+ 105us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.21.51.190:38831: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33535/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":97,"threads_started":1}
I20260809 16:20:42.868494 21710 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrNqUCn/build/release/bin/kudu with pid 30036
E20260809 16:20:42.882181 30185 webserver.cc:533] Webserver: sslize failed:
E20260809 16:20:42.882277 30185 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
2026-08-09T16:20:42Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (235 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-08-09T16:20:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T16:20:42Z Disabled control of system clock
I20260809 16:20:42.919857 21710 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrNqUCn/build/release/bin/kudu
/tmp/dist-test-taskrNqUCn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786292417722232-21710-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786292417722232-21710-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786292417722232-21710-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786292417722232-21710-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.51.190:36895
--webserver_interface=127.21.51.190
--webserver_port=0
--builtin_ntp_servers=127.21.51.148:35207
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.51.190:36895
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786292417722232-21710-0/testchainca.cert with env {}
W20260809 16:20:43.002163 30190 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:20:43.002377 30190 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:20:43.002399 30190 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:20:43.003616 30190 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 16:20:43.003665 30190 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:20:43.003679 30190 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:20:43.003692 30190 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 16:20:43.003705 30190 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260809 16:20:43.003755 30190 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260809 16:20:43.003777 30190 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260809 16:20:43.005321 30190 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.51.148:35207
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786292417722232-21710-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786292417722232-21710-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.51.190:36895
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.51.190:36895
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786292417722232-21710-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.51.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786292417722232-21710-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30190
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786292417722232-21710-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:20:43.005532 30190 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:20:43.005708 30190 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:20:43.010851 30198 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:20:43.010910 30190 server_base.cc:1034] running on GCE node
W20260809 16:20:43.010887 30200 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:20:43.010874 30196 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:20:43.011274 30190 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:20:43.011624 30190 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:20:43.012781 30190 hybrid_clock.cc:648] HybridClock initialized: now 1786292443012756 us; error 45 us; skew 500 ppm
I20260809 16:20:43.014164 30190 webserver.cc:460] Webserver started at http://127.21.51.190:34987/ using document root <none> and password file <none>
I20260809 16:20:43.014449 30190 fs_manager.cc:359] Metadata directory not provided
I20260809 16:20:43.014523 30190 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:20:43.014672 30190 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:20:43.016261 30190 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786292417722232-21710-0/minicluster-data/master-0/data/instance:
uuid: "878917615fa54be69174ec572f1f903c"
format_stamp: "Formatted at 2026-08-09 16:20:43 on dist-test-slave-s11t"
server_key: "bf1d322fce7a53b54db73a42f4c35f77"
server_key_iv: "5dda7e430a7cd6e038eb1504eb90a932"
server_key_version: "encryptionkey@0"
I20260809 16:20:43.016644 30190 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786292417722232-21710-0/minicluster-data/master-0/wal/instance:
uuid: "878917615fa54be69174ec572f1f903c"
format_stamp: "Formatted at 2026-08-09 16:20:43 on dist-test-slave-s11t"
server_key: "bf1d322fce7a53b54db73a42f4c35f77"
server_key_iv: "5dda7e430a7cd6e038eb1504eb90a932"
server_key_version: "encryptionkey@0"
I20260809 16:20:43.018067 30190 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260809 16:20:43.018997 30210 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:20:43.019166 30190 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 16:20:43.019277 30190 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786292417722232-21710-0/minicluster-data/master-0/data,/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786292417722232-21710-0/minicluster-data/master-0/wal
uuid: "878917615fa54be69174ec572f1f903c"
format_stamp: "Formatted at 2026-08-09 16:20:43 on dist-test-slave-s11t"
server_key: "bf1d322fce7a53b54db73a42f4c35f77"
server_key_iv: "5dda7e430a7cd6e038eb1504eb90a932"
server_key_version: "encryptionkey@0"
I20260809 16:20:43.019361 30190 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786292417722232-21710-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786292417722232-21710-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786292417722232-21710-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:20:43.039490 30190 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:20:43.039858 30190 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:20:43.040028 30190 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:20:43.044391 30190 rpc_server.cc:307] RPC server started. Bound to: 127.21.51.190:36895
I20260809 16:20:43.044437 30294 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.51.190:36895 every 8 connection(s)
I20260809 16:20:43.044844 30190 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786292417722232-21710-0/minicluster-data/master-0/data/info.pb
I20260809 16:20:43.045907 30295 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:20:43.048367 30295 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 878917615fa54be69174ec572f1f903c: Bootstrap starting.
I20260809 16:20:43.049000 30295 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 878917615fa54be69174ec572f1f903c: Neither blocks nor log segments found. Creating new log.
I20260809 16:20:43.049307 30295 log.cc:824] T 00000000000000000000000000000000 P 878917615fa54be69174ec572f1f903c: Log is configured to *not* fsync() on all Append() calls
I20260809 16:20:43.050097 30295 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 878917615fa54be69174ec572f1f903c: No bootstrap required, opened a new log
I20260809 16:20:43.051693 30295 raft_consensus.cc:348] T 00000000000000000000000000000000 P 878917615fa54be69174ec572f1f903c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "878917615fa54be69174ec572f1f903c" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 36895 } }
I20260809 16:20:43.051843 30295 raft_consensus.cc:374] T 00000000000000000000000000000000 P 878917615fa54be69174ec572f1f903c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:20:43.051867 30295 raft_consensus.cc:729] T 00000000000000000000000000000000 P 878917615fa54be69174ec572f1f903c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 878917615fa54be69174ec572f1f903c, State: Initialized, Role: FOLLOWER
I20260809 16:20:43.052191 30295 consensus_queue.cc:260] T 00000000000000000000000000000000 P 878917615fa54be69174ec572f1f903c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "878917615fa54be69174ec572f1f903c" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 36895 } }
I20260809 16:20:43.052296 30295 raft_consensus.cc:388] T 00000000000000000000000000000000 P 878917615fa54be69174ec572f1f903c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 16:20:43.052321 30295 raft_consensus.cc:482] T 00000000000000000000000000000000 P 878917615fa54be69174ec572f1f903c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 16:20:43.052389 30295 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 878917615fa54be69174ec572f1f903c [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:20:43.052989 30295 raft_consensus.cc:504] T 00000000000000000000000000000000 P 878917615fa54be69174ec572f1f903c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "878917615fa54be69174ec572f1f903c" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 36895 } }
I20260809 16:20:43.053099 30295 leader_election.cc:302] T 00000000000000000000000000000000 P 878917615fa54be69174ec572f1f903c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 878917615fa54be69174ec572f1f903c; no voters:
I20260809 16:20:43.053301 30295 leader_election.cc:288] T 00000000000000000000000000000000 P 878917615fa54be69174ec572f1f903c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 16:20:43.053364 30298 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 878917615fa54be69174ec572f1f903c [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:20:43.053555 30298 raft_consensus.cc:686] T 00000000000000000000000000000000 P 878917615fa54be69174ec572f1f903c [term 1 LEADER]: Becoming Leader. State: Replica: 878917615fa54be69174ec572f1f903c, State: Running, Role: LEADER
I20260809 16:20:43.053568 30295 sys_catalog.cc:564] T 00000000000000000000000000000000 P 878917615fa54be69174ec572f1f903c [sys.catalog]: configured and running, proceeding with master startup.
I20260809 16:20:43.053666 30298 consensus_queue.cc:237] T 00000000000000000000000000000000 P 878917615fa54be69174ec572f1f903c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "878917615fa54be69174ec572f1f903c" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 36895 } }
I20260809 16:20:43.054113 30300 sys_catalog.cc:455] T 00000000000000000000000000000000 P 878917615fa54be69174ec572f1f903c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 878917615fa54be69174ec572f1f903c. Latest consensus state: current_term: 1 leader_uuid: "878917615fa54be69174ec572f1f903c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "878917615fa54be69174ec572f1f903c" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 36895 } } }
I20260809 16:20:43.054203 30300 sys_catalog.cc:458] T 00000000000000000000000000000000 P 878917615fa54be69174ec572f1f903c [sys.catalog]: This master's current role is: LEADER
I20260809 16:20:43.054307 30299 sys_catalog.cc:455] T 00000000000000000000000000000000 P 878917615fa54be69174ec572f1f903c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "878917615fa54be69174ec572f1f903c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "878917615fa54be69174ec572f1f903c" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 36895 } } }
I20260809 16:20:43.054442 30299 sys_catalog.cc:458] T 00000000000000000000000000000000 P 878917615fa54be69174ec572f1f903c [sys.catalog]: This master's current role is: LEADER
I20260809 16:20:43.054754 21710 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrNqUCn/build/release/bin/kudu as pid 30190
I20260809 16:20:43.054867 21710 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786292417722232-21710-0/minicluster-data/master-0/wal/instance
I20260809 16:20:43.055074 21710 external_mini_cluster.cc:1428] Setting key 95371805e450799f679d1068dee9755d
W20260809 16:20:43.055538 30325 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 878917615fa54be69174ec572f1f903c: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 16:20:43.055593 30325 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 16:20:43.055683 30312 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 16:20:43.055913 30312 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 16:20:43.057463 30312 catalog_manager.cc:1347] Generated new cluster ID: 82af77d2052c4b5ebd8951c9ed50a247
I20260809 16:20:43.057518 30312 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 16:20:43.064016 30329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:43.055533 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52350 (local address 127.21.51.190:36895)
0809 16:20:43.056661 (+ 1128us) server_negotiation.cc:206] Beginning negotiation
0809 16:20:43.056668 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 16:20:43.056739 (+ 71us) server_negotiation.cc:407] Connection header received
0809 16:20:43.056866 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:20:43.056873 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:20:43.056953 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:20:43.057108 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:20:43.057462 (+ 354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:43.058890 (+ 1428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:43.061778 (+ 2888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:43.062441 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:43.062572 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:43.063309 (+ 737us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:20:43.063313 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:20:43.063313 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 16:20:43.063530 (+ 217us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:20:43.063574 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 16:20:43.063588 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:20:43.063663 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0809 16:20:43.063731 (+ 68us) server_negotiation.cc:299] Negotiation successful
0809 16:20:43.063812 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1066,"thread_start_us":89,"threads_started":1}
I20260809 16:20:43.064945 21710 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 16:20:43.075456 30312 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 16:20:43.076170 30312 catalog_manager.cc:1504] Loading token signing keys...
I20260809 16:20:43.087586 30312 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 878917615fa54be69174ec572f1f903c: Generated new TSK 0
I20260809 16:20:43.087795 30312 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 16:20:43.956933 30329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:43.097146 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52364 (local address 127.21.51.190:36895)
0809 16:20:43.097215 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0809 16:20:43.097218 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 16:20:43.097366 (+ 148us) server_negotiation.cc:407] Connection header received
0809 16:20:43.097398 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:20:43.097399 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:20:43.097421 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:20:43.097483 (+ 62us) server_negotiation.cc:226] Negotiated authn=JWT
0809 16:20:43.097644 (+ 161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:43.098166 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:43.099309 (+ 1143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:43.099491 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:43.099548 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:43.099753 (+ 205us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0809 16:20:43.956547 (+856794us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0809 16:20:43.956854 (+ 307us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:52364: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":43}
W20260809 16:20:43.956962 30340 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:20:43.097089 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:36895 (local address 127.0.0.1:52364)
0809 16:20:43.097252 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0809 16:20:43.097272 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 16:20:43.097339 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:20:43.097501 (+ 162us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:20:43.097502 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:20:43.097504 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0809 16:20:43.097565 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:20:43.097568 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:20:43.098246 (+ 678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:20:43.098247 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:20:43.099209 (+ 962us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:20:43.099212 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:20:43.099575 (+ 363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:20:43.099576 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:20:43.099640 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:43.099646 (+ 6us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0809 16:20:43.956728 (+857082us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0809 16:20:43.956878 (+ 150us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.21.51.190:36895: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":113,"threads_started":1}
W20260809 16:20:43.957031 30329 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:52364: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
I20260809 16:20:43.957374 21710 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrNqUCn/build/release/bin/kudu with pid 30190
2026-08-09T16:20:43Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (1068 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (91 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (90 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786292417722232-21710-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 16:20:44 dist-test-slave-s11t krb5kdc[30364](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 16:20:44 dist-test-slave-s11t krb5kdc[30364](info): set up 2 sockets
Aug 09 16:20:44 dist-test-slave-s11t krb5kdc[30364](info): commencing operation
krb5kdc: starting...
W20260809 16:20:45.374392 21710 mini_kdc.cc:121] Time spent starting KDC: real 1.213s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786292417722232-21710-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786292417722232-21710-0/krb5kdc/test-user.keytab.
Aug 09 16:20:45 dist-test-slave-s11t krb5kdc[30364](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292445, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T16:20:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T16:20:45Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.51.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.51.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.51.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.190.keytab.
Entry for principal kudu/127.21.51.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.190.keytab.
I20260809 16:20:45.495203 21710 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrNqUCn/build/release/bin/kudu
/tmp/dist-test-taskrNqUCn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786292417722232-21710-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786292417722232-21710-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786292417722232-21710-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786292417722232-21710-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.51.190:45603
--webserver_interface=127.21.51.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.190.keytab
--principal=kudu/127.21.51.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.51.148:37127
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.51.190:45603
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786292417722232-21710-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786292417722232-21710-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786292417722232-21710-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:20:45.577529 30383 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:20:45.577769 30383 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:20:45.577804 30383 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:20:45.578962 30383 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 16:20:45.579028 30383 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:20:45.579053 30383 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:20:45.579073 30383 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 16:20:45.579103 30383 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 16:20:45.580597 30383 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.51.148:37127
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786292417722232-21710-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786292417722232-21710-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.51.190:45603
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.190.keytab
--principal=kudu/127.21.51.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.51.190:45603
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786292417722232-21710-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.51.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30383
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786292417722232-21710-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:20:45.580910 30383 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:20:45.581142 30383 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:20:45.586359 30392 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:20:45.586362 30389 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:20:45.586365 30390 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:20:45.586396 30383 server_base.cc:1034] running on GCE node
I20260809 16:20:45.586865 30383 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:20:45.587139 30383 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:20:45.588310 30383 hybrid_clock.cc:648] HybridClock initialized: now 1786292445588281 us; error 41 us; skew 500 ppm
Aug 09 16:20:45 dist-test-slave-s11t krb5kdc[30364](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292445, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.51.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:20:45.591109 30383 init.cc:377] Logged in from keytab as kudu/127.21.51.190@KRBTEST.COM (short username kudu)
I20260809 16:20:45.591554 30383 webserver.cc:460] Webserver started at http://127.21.51.190:34773/ using document root <none> and password file <none>
I20260809 16:20:45.591804 30383 fs_manager.cc:359] Metadata directory not provided
I20260809 16:20:45.591876 30383 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:20:45.592025 30383 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:20:45.593160 30383 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786292417722232-21710-0/minicluster-data/master-0/data/instance:
uuid: "9676beb08a604f4786186ffa65cb6fb6"
format_stamp: "Formatted at 2026-08-09 16:20:45 on dist-test-slave-s11t"
server_key: "162cdf288628c0638b9228778ba5bca8"
server_key_iv: "d9d29a554bef6e4a887553cc75473fd9"
server_key_version: "encryptionkey@0"
I20260809 16:20:45.593530 30383 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786292417722232-21710-0/minicluster-data/master-0/wal/instance:
uuid: "9676beb08a604f4786186ffa65cb6fb6"
format_stamp: "Formatted at 2026-08-09 16:20:45 on dist-test-slave-s11t"
server_key: "162cdf288628c0638b9228778ba5bca8"
server_key_iv: "d9d29a554bef6e4a887553cc75473fd9"
server_key_version: "encryptionkey@0"
I20260809 16:20:45.594915 30383 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 16:20:45.595609 30404 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:20:45.595824 30383 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 16:20:45.595937 30383 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786292417722232-21710-0/minicluster-data/master-0/data,/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786292417722232-21710-0/minicluster-data/master-0/wal
uuid: "9676beb08a604f4786186ffa65cb6fb6"
format_stamp: "Formatted at 2026-08-09 16:20:45 on dist-test-slave-s11t"
server_key: "162cdf288628c0638b9228778ba5bca8"
server_key_iv: "d9d29a554bef6e4a887553cc75473fd9"
server_key_version: "encryptionkey@0"
I20260809 16:20:45.596020 30383 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786292417722232-21710-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786292417722232-21710-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786292417722232-21710-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:20:45.619397 30383 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:20:45.621821 30383 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:20:45.622004 30383 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:20:45.626093 30383 rpc_server.cc:307] RPC server started. Bound to: 127.21.51.190:45603
I20260809 16:20:45.626151 30487 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.51.190:45603 every 8 connection(s)
I20260809 16:20:45.626602 30383 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786292417722232-21710-0/minicluster-data/master-0/data/info.pb
I20260809 16:20:45.627758 30488 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:20:45.630066 21710 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrNqUCn/build/release/bin/kudu as pid 30383
I20260809 16:20:45.630148 21710 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786292417722232-21710-0/minicluster-data/master-0/wal/instance
I20260809 16:20:45.630271 30488 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9676beb08a604f4786186ffa65cb6fb6: Bootstrap starting.
I20260809 16:20:45.630380 21710 external_mini_cluster.cc:1428] Setting key 3c06f502ac02ea49a1b8025da18f9682
I20260809 16:20:45.631024 30488 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9676beb08a604f4786186ffa65cb6fb6: Neither blocks nor log segments found. Creating new log.
I20260809 16:20:45.631387 30488 log.cc:824] T 00000000000000000000000000000000 P 9676beb08a604f4786186ffa65cb6fb6: Log is configured to *not* fsync() on all Append() calls
I20260809 16:20:45.632165 30488 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9676beb08a604f4786186ffa65cb6fb6: No bootstrap required, opened a new log
I20260809 16:20:45.634555 30488 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9676beb08a604f4786186ffa65cb6fb6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9676beb08a604f4786186ffa65cb6fb6" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 45603 } }
I20260809 16:20:45.634689 30488 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9676beb08a604f4786186ffa65cb6fb6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:20:45.634712 30488 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9676beb08a604f4786186ffa65cb6fb6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9676beb08a604f4786186ffa65cb6fb6, State: Initialized, Role: FOLLOWER
I20260809 16:20:45.634855 30488 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9676beb08a604f4786186ffa65cb6fb6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9676beb08a604f4786186ffa65cb6fb6" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 45603 } }
I20260809 16:20:45.634932 30488 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9676beb08a604f4786186ffa65cb6fb6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 16:20:45.634955 30488 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9676beb08a604f4786186ffa65cb6fb6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 16:20:45.635025 30488 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9676beb08a604f4786186ffa65cb6fb6 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:20:45.635569 30488 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9676beb08a604f4786186ffa65cb6fb6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9676beb08a604f4786186ffa65cb6fb6" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 45603 } }
I20260809 16:20:45.635694 30488 leader_election.cc:302] T 00000000000000000000000000000000 P 9676beb08a604f4786186ffa65cb6fb6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9676beb08a604f4786186ffa65cb6fb6; no voters:
I20260809 16:20:45.635864 30488 leader_election.cc:288] T 00000000000000000000000000000000 P 9676beb08a604f4786186ffa65cb6fb6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 16:20:45.635900 30497 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9676beb08a604f4786186ffa65cb6fb6 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:20:45.636044 30497 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9676beb08a604f4786186ffa65cb6fb6 [term 1 LEADER]: Becoming Leader. State: Replica: 9676beb08a604f4786186ffa65cb6fb6, State: Running, Role: LEADER
I20260809 16:20:45.636142 30488 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9676beb08a604f4786186ffa65cb6fb6 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 16:20:45.636127 30497 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9676beb08a604f4786186ffa65cb6fb6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9676beb08a604f4786186ffa65cb6fb6" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 45603 } }
I20260809 16:20:45.636579 30499 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9676beb08a604f4786186ffa65cb6fb6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9676beb08a604f4786186ffa65cb6fb6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9676beb08a604f4786186ffa65cb6fb6" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 45603 } } }
I20260809 16:20:45.636715 30499 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9676beb08a604f4786186ffa65cb6fb6 [sys.catalog]: This master's current role is: LEADER
I20260809 16:20:45.636570 30497 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9676beb08a604f4786186ffa65cb6fb6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9676beb08a604f4786186ffa65cb6fb6. Latest consensus state: current_term: 1 leader_uuid: "9676beb08a604f4786186ffa65cb6fb6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9676beb08a604f4786186ffa65cb6fb6" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 45603 } } }
I20260809 16:20:45.637022 30497 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9676beb08a604f4786186ffa65cb6fb6 [sys.catalog]: This master's current role is: LEADER
I20260809 16:20:45.637439 30508 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 16:20:45.637974 30508 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 09 16:20:45 dist-test-slave-s11t krb5kdc[30364](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292445, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.51.190@KRBTEST.COM
I20260809 16:20:45.640332 30508 catalog_manager.cc:1347] Generated new cluster ID: 5469922922d746829fc199009e483cfe
I20260809 16:20:45.640398 30508 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 16:20:45.642457 30492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:45.630932 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53244 (local address 127.21.51.190:45603)
0809 16:20:45.631140 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0809 16:20:45.631149 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 16:20:45.631189 (+ 40us) server_negotiation.cc:407] Connection header received
0809 16:20:45.631765 (+ 576us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:20:45.631773 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:20:45.631853 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:20:45.632006 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:20:45.632670 (+ 664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:45.634140 (+ 1470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:45.635042 (+ 902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:45.635494 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:45.635627 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:45.638562 (+ 2935us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:20:45.638566 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:20:45.638566 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:20:45.638578 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:20:45.640637 (+ 2059us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:20:45.641204 (+ 567us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:20:45.641207 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:20:45.641208 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:20:45.641311 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:20:45.641547 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:20:45.641548 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:20:45.641548 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:20:45.641836 (+ 288us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:20:45.641975 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0809 16:20:45.642124 (+ 149us) server_negotiation.cc:299] Negotiation successful
0809 16:20:45.642252 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":156,"thread_start_us":86,"threads_started":1}
I20260809 16:20:45.667382 30508 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 16:20:45.668020 30508 catalog_manager.cc:1504] Loading token signing keys...
I20260809 16:20:45.671052 30508 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9676beb08a604f4786186ffa65cb6fb6: Generated new TSK 0
I20260809 16:20:45.671260 30508 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.21.51.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.51.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.51.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.129.keytab.
Entry for principal kudu/127.21.51.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.129.keytab.
I20260809 16:20:45.701733 21710 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrNqUCn/build/release/bin/kudu
/tmp/dist-test-taskrNqUCn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786292417722232-21710-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786292417722232-21710-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786292417722232-21710-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786292417722232-21710-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.51.129:0
--local_ip_for_outbound_sockets=127.21.51.129
--webserver_interface=127.21.51.129
--webserver_port=0
--tserver_master_addrs=127.21.51.190:45603
--keytab_file=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.129.keytab
--principal=kudu/127.21.51.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.51.148:37127
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786292417722232-21710-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786292417722232-21710-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786292417722232-21710-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:20:45.783787 30533 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:20:45.783980 30533 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:20:45.784003 30533 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:20:45.785163 30533 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:20:45.785200 30533 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:20:45.785254 30533 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.51.129
I20260809 16:20:45.786787 30533 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.51.148:37127
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786292417722232-21710-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786292417722232-21710-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.129.keytab
--principal=kudu/127.21.51.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.51.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786292417722232-21710-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.51.129
--webserver_port=0
--tserver_master_addrs=127.21.51.190:45603
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30533
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.51.129
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786292417722232-21710-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:20:45.787036 30533 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:20:45.787272 30533 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:20:45.792364 30542 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:20:45.792415 30533 server_base.cc:1034] running on GCE node
W20260809 16:20:45.792387 30539 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:20:45.792387 30544 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:20:45.792920 30533 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:20:45.793123 30533 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:20:45.794262 30533 hybrid_clock.cc:648] HybridClock initialized: now 1786292445794232 us; error 49 us; skew 500 ppm
Aug 09 16:20:45 dist-test-slave-s11t krb5kdc[30364](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292445, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.51.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:20:45.799288 30533 init.cc:377] Logged in from keytab as kudu/127.21.51.129@KRBTEST.COM (short username kudu)
I20260809 16:20:45.799798 30533 webserver.cc:460] Webserver started at http://127.21.51.129:38637/ using document root <none> and password file <none>
I20260809 16:20:45.800076 30533 fs_manager.cc:359] Metadata directory not provided
I20260809 16:20:45.800123 30533 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:20:45.800212 30533 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:20:45.801288 30533 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786292417722232-21710-0/minicluster-data/ts-0/data/instance:
uuid: "160b139042384e6da70db2301763af42"
format_stamp: "Formatted at 2026-08-09 16:20:45 on dist-test-slave-s11t"
server_key: "b242a02a0ed329b83969f0274520ba69"
server_key_iv: "06e3c4af0963d18ef3b22087d17d2fb7"
server_key_version: "encryptionkey@0"
I20260809 16:20:45.801589 30533 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786292417722232-21710-0/minicluster-data/ts-0/wal/instance:
uuid: "160b139042384e6da70db2301763af42"
format_stamp: "Formatted at 2026-08-09 16:20:45 on dist-test-slave-s11t"
server_key: "b242a02a0ed329b83969f0274520ba69"
server_key_iv: "06e3c4af0963d18ef3b22087d17d2fb7"
server_key_version: "encryptionkey@0"
I20260809 16:20:45.802978 30533 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260809 16:20:45.803717 30556 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:20:45.803956 30533 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 16:20:45.804072 30533 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786292417722232-21710-0/minicluster-data/ts-0/data,/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786292417722232-21710-0/minicluster-data/ts-0/wal
uuid: "160b139042384e6da70db2301763af42"
format_stamp: "Formatted at 2026-08-09 16:20:45 on dist-test-slave-s11t"
server_key: "b242a02a0ed329b83969f0274520ba69"
server_key_iv: "06e3c4af0963d18ef3b22087d17d2fb7"
server_key_version: "encryptionkey@0"
I20260809 16:20:45.804159 30533 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786292417722232-21710-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786292417722232-21710-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786292417722232-21710-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:20:45.811625 30533 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:20:45.813553 30533 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:20:45.813722 30533 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:20:45.813961 30533 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:20:45.814306 30533 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:20:45.814376 30533 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:20:45.814424 30533 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:20:45.814455 30533 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:20:45.820889 30533 rpc_server.cc:307] RPC server started. Bound to: 127.21.51.129:41679
I20260809 16:20:45.820945 30727 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.51.129:41679 every 8 connection(s)
I20260809 16:20:45.821267 30533 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786292417722232-21710-0/minicluster-data/ts-0/data/info.pb
I20260809 16:20:45.827086 21710 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrNqUCn/build/release/bin/kudu as pid 30533
I20260809 16:20:45.827229 21710 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786292417722232-21710-0/minicluster-data/ts-0/wal/instance
I20260809 16:20:45.827483 21710 external_mini_cluster.cc:1428] Setting key 98688a0024f903921343da0d6f0a9043
Aug 09 16:20:45 dist-test-slave-s11t krb5kdc[30364](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292445, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.51.129@KRBTEST.COM for kudu/127.21.51.190@KRBTEST.COM
I20260809 16:20:45.831343 30492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:45.822020 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.51.129:54913 (local address 127.21.51.190:45603)
0809 16:20:45.822096 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0809 16:20:45.822099 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 16:20:45.822307 (+ 208us) server_negotiation.cc:407] Connection header received
0809 16:20:45.823223 (+ 916us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:20:45.823224 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:20:45.823235 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:20:45.823301 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:20:45.823882 (+ 581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:45.824448 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:45.825525 (+ 1077us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:45.825772 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:45.825838 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:45.828239 (+ 2401us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:20:45.828241 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:20:45.828242 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:20:45.828251 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:20:45.829762 (+ 1511us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:20:45.830264 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:20:45.830266 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:20:45.830266 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:20:45.830337 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:20:45.830615 (+ 278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:20:45.830615 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:20:45.830616 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:20:45.830748 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:20:45.830830 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0809 16:20:45.831137 (+ 307us) server_negotiation.cc:299] Negotiation successful
0809 16:20:45.831245 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260809 16:20:45.831545 30731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:45.822004 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:45603 (local address 127.21.51.129:54913)
0809 16:20:45.822183 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0809 16:20:45.822219 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0809 16:20:45.823128 (+ 909us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:20:45.823355 (+ 227us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:20:45.823358 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:20:45.823613 (+ 255us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:20:45.823797 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:20:45.823801 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:20:45.824532 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:20:45.824533 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:20:45.825432 (+ 899us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:20:45.825434 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:20:45.825854 (+ 420us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:20:45.825855 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:20:45.825967 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:45.826459 (+ 492us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:20:45.826466 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:20:45.828127 (+ 1661us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:20:45.829858 (+ 1731us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:20:45.829865 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:20:45.829867 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:20:45.830115 (+ 248us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:20:45.830415 (+ 300us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:20:45.830416 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:20:45.830416 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:20:45.830531 (+ 115us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:20:45.830834 (+ 303us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:20:45.830841 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:20:45.830985 (+ 144us) client_negotiation.cc:769] Sending connection context
0809 16:20:45.831098 (+ 113us) client_negotiation.cc:240] Negotiation successful
0809 16:20:45.831283 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":103,"threads_started":1}
I20260809 16:20:45.832093 30728 heartbeater.cc:347] Connected to a master server at 127.21.51.190:45603
I20260809 16:20:45.832190 30728 heartbeater.cc:464] Registering TS with master...
I20260809 16:20:45.832427 30728 heartbeater.cc:511] Master 127.21.51.190:45603 requested a full tablet report, sending...
I20260809 16:20:45.833091 30438 ts_manager.cc:194] Registered new tserver with Master: 160b139042384e6da70db2301763af42 (127.21.51.129:41679)
I20260809 16:20:45.834123 30438 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.51.129@KRBTEST.COM'} at 127.21.51.129:54913
WARNING: no policy specified for kudu/127.21.51.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.51.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.51.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.130.keytab.
Entry for principal kudu/127.21.51.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.130.keytab.
I20260809 16:20:45.852748 21710 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrNqUCn/build/release/bin/kudu
/tmp/dist-test-taskrNqUCn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786292417722232-21710-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786292417722232-21710-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786292417722232-21710-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786292417722232-21710-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.51.130:0
--local_ip_for_outbound_sockets=127.21.51.130
--webserver_interface=127.21.51.130
--webserver_port=0
--tserver_master_addrs=127.21.51.190:45603
--keytab_file=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.130.keytab
--principal=kudu/127.21.51.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.51.148:37127
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786292417722232-21710-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786292417722232-21710-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786292417722232-21710-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:20:45.933425 30734 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:20:45.933624 30734 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:20:45.933646 30734 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:20:45.934724 30734 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:20:45.934758 30734 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:20:45.934783 30734 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.51.130
I20260809 16:20:45.936292 30734 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.51.148:37127
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786292417722232-21710-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786292417722232-21710-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.130.keytab
--principal=kudu/127.21.51.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.51.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786292417722232-21710-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.51.130
--webserver_port=0
--tserver_master_addrs=127.21.51.190:45603
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30734
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.51.130
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786292417722232-21710-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:20:45.936473 30734 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:20:45.936739 30734 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:20:45.942214 30746 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:20:45.942226 30745 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:20:45.942230 30749 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:20:45.942516 30734 server_base.cc:1034] running on GCE node
I20260809 16:20:45.942721 30734 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:20:45.942991 30734 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:20:45.944162 30734 hybrid_clock.cc:648] HybridClock initialized: now 1786292445944139 us; error 36 us; skew 500 ppm
Aug 09 16:20:45 dist-test-slave-s11t krb5kdc[30364](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292445, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.51.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:20:45.946722 30734 init.cc:377] Logged in from keytab as kudu/127.21.51.130@KRBTEST.COM (short username kudu)
I20260809 16:20:45.947132 30734 webserver.cc:460] Webserver started at http://127.21.51.130:34431/ using document root <none> and password file <none>
I20260809 16:20:45.947391 30734 fs_manager.cc:359] Metadata directory not provided
I20260809 16:20:45.947439 30734 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:20:45.947580 30734 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:20:45.948613 30734 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786292417722232-21710-0/minicluster-data/ts-1/data/instance:
uuid: "1194cc732e524909957cd9a75bbeb2ee"
format_stamp: "Formatted at 2026-08-09 16:20:45 on dist-test-slave-s11t"
server_key: "fadb644077f5442c254e0633b4d27a8d"
server_key_iv: "989c31b7363acd4115aa14ff6dee1331"
server_key_version: "encryptionkey@0"
I20260809 16:20:45.949039 30734 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786292417722232-21710-0/minicluster-data/ts-1/wal/instance:
uuid: "1194cc732e524909957cd9a75bbeb2ee"
format_stamp: "Formatted at 2026-08-09 16:20:45 on dist-test-slave-s11t"
server_key: "fadb644077f5442c254e0633b4d27a8d"
server_key_iv: "989c31b7363acd4115aa14ff6dee1331"
server_key_version: "encryptionkey@0"
I20260809 16:20:45.950377 30734 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:20:45.951113 30759 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:20:45.951327 30734 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:20:45.951406 30734 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786292417722232-21710-0/minicluster-data/ts-1/data,/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786292417722232-21710-0/minicluster-data/ts-1/wal
uuid: "1194cc732e524909957cd9a75bbeb2ee"
format_stamp: "Formatted at 2026-08-09 16:20:45 on dist-test-slave-s11t"
server_key: "fadb644077f5442c254e0633b4d27a8d"
server_key_iv: "989c31b7363acd4115aa14ff6dee1331"
server_key_version: "encryptionkey@0"
I20260809 16:20:45.951506 30734 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786292417722232-21710-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786292417722232-21710-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786292417722232-21710-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:20:45.977977 30734 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:20:45.979913 30734 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:20:45.980079 30734 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:20:45.980334 30734 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:20:45.980664 30734 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:20:45.980767 30734 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:20:45.980813 30734 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:20:45.980839 30734 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:20:45.986749 30734 rpc_server.cc:307] RPC server started. Bound to: 127.21.51.130:44773
I20260809 16:20:45.986796 30951 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.51.130:44773 every 8 connection(s)
I20260809 16:20:45.987115 30734 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786292417722232-21710-0/minicluster-data/ts-1/data/info.pb
I20260809 16:20:45.987299 21710 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrNqUCn/build/release/bin/kudu as pid 30734
I20260809 16:20:45.987416 21710 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786292417722232-21710-0/minicluster-data/ts-1/wal/instance
I20260809 16:20:45.987672 21710 external_mini_cluster.cc:1428] Setting key d0f14e6a5ddf6e060f642c199ef850a7
Aug 09 16:20:45 dist-test-slave-s11t krb5kdc[30364](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292445, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.51.130@KRBTEST.COM for kudu/127.21.51.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.21.51.131@KRBTEST.COM; defaulting to no policy
I20260809 16:20:45.999385 30492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:45.987995 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.51.130:52703 (local address 127.21.51.190:45603)
0809 16:20:45.988043 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0809 16:20:45.988046 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 16:20:45.988309 (+ 263us) server_negotiation.cc:407] Connection header received
0809 16:20:45.989635 (+ 1326us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:20:45.989636 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:20:45.989646 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:20:45.989700 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:20:45.990582 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:45.991484 (+ 902us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:45.993139 (+ 1655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:45.993366 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:45.993440 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:45.996039 (+ 2599us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:20:45.996041 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:20:45.996041 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:20:45.996049 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:20:45.997717 (+ 1668us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:20:45.998229 (+ 512us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:20:45.998230 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:20:45.998231 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:20:45.998304 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:20:45.998568 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:20:45.998569 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:20:45.998570 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:20:45.998734 (+ 164us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:20:45.998814 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0809 16:20:45.999225 (+ 411us) server_negotiation.cc:299] Negotiation successful
0809 16:20:45.999316 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260809 16:20:45.999652 30956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:45.987973 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:45603 (local address 127.21.51.130:52703)
0809 16:20:45.988142 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0809 16:20:45.988193 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0809 16:20:45.989521 (+ 1328us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:20:45.989772 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:20:45.989778 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:20:45.990173 (+ 395us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:20:45.990471 (+ 298us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:20:45.990477 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:20:45.991590 (+ 1113us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:20:45.991591 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:20:45.993034 (+ 1443us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:20:45.993037 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:20:45.993487 (+ 450us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:20:45.993489 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:20:45.993648 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:45.994189 (+ 541us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:20:45.994201 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:20:45.995939 (+ 1738us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:20:45.997824 (+ 1885us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:20:45.997830 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:20:45.997832 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:20:45.998147 (+ 315us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:20:45.998382 (+ 235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:20:45.998383 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:20:45.998383 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:20:45.998508 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:20:45.998801 (+ 293us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:20:45.998808 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:20:45.999014 (+ 206us) client_negotiation.cc:769] Sending connection context
0809 16:20:45.999138 (+ 124us) client_negotiation.cc:240] Negotiation successful
0809 16:20:45.999358 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":90,"threads_started":1}
I20260809 16:20:46.000182 30953 heartbeater.cc:347] Connected to a master server at 127.21.51.190:45603
I20260809 16:20:46.000274 30953 heartbeater.cc:464] Registering TS with master...
I20260809 16:20:46.000425 30953 heartbeater.cc:511] Master 127.21.51.190:45603 requested a full tablet report, sending...
I20260809 16:20:46.000849 30438 ts_manager.cc:194] Registered new tserver with Master: 1194cc732e524909957cd9a75bbeb2ee (127.21.51.130:44773)
I20260809 16:20:46.001377 30438 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.51.130@KRBTEST.COM'} at 127.21.51.130:52703
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.51.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.51.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.131.keytab.
Entry for principal kudu/127.21.51.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.131.keytab.
I20260809 16:20:46.019260 21710 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrNqUCn/build/release/bin/kudu
/tmp/dist-test-taskrNqUCn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786292417722232-21710-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786292417722232-21710-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786292417722232-21710-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786292417722232-21710-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.51.131:0
--local_ip_for_outbound_sockets=127.21.51.131
--webserver_interface=127.21.51.131
--webserver_port=0
--tserver_master_addrs=127.21.51.190:45603
--keytab_file=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.131.keytab
--principal=kudu/127.21.51.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.51.148:37127
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786292417722232-21710-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786292417722232-21710-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786292417722232-21710-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:20:46.103632 30959 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:20:46.103843 30959 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:20:46.103866 30959 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:20:46.105079 30959 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:20:46.105124 30959 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:20:46.105150 30959 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.51.131
I20260809 16:20:46.106637 30959 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.51.148:37127
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786292417722232-21710-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786292417722232-21710-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.131.keytab
--principal=kudu/127.21.51.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.51.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786292417722232-21710-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.51.131
--webserver_port=0
--tserver_master_addrs=127.21.51.190:45603
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30959
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.51.131
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786292417722232-21710-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:20:46.106895 30959 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:20:46.107136 30959 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:20:46.112370 30970 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:20:46.112386 30966 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:20:46.112386 30965 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:20:46.112764 30959 server_base.cc:1034] running on GCE node
I20260809 16:20:46.113008 30959 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:20:46.113243 30959 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:20:46.114440 30959 hybrid_clock.cc:648] HybridClock initialized: now 1786292446114428 us; error 36 us; skew 500 ppm
Aug 09 16:20:46 dist-test-slave-s11t krb5kdc[30364](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292446, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.51.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:20:46.117161 30959 init.cc:377] Logged in from keytab as kudu/127.21.51.131@KRBTEST.COM (short username kudu)
I20260809 16:20:46.117605 30959 webserver.cc:460] Webserver started at http://127.21.51.131:37635/ using document root <none> and password file <none>
I20260809 16:20:46.117856 30959 fs_manager.cc:359] Metadata directory not provided
I20260809 16:20:46.117930 30959 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:20:46.118077 30959 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:20:46.119190 30959 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786292417722232-21710-0/minicluster-data/ts-2/data/instance:
uuid: "7c41c4e4decc4678b69c8506f0d7ad44"
format_stamp: "Formatted at 2026-08-09 16:20:46 on dist-test-slave-s11t"
server_key: "b1b6ae93f0803893fd6c4cf0553d50bd"
server_key_iv: "b17d0d98dde063f81b0f917ad28a3e22"
server_key_version: "encryptionkey@0"
I20260809 16:20:46.119565 30959 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786292417722232-21710-0/minicluster-data/ts-2/wal/instance:
uuid: "7c41c4e4decc4678b69c8506f0d7ad44"
format_stamp: "Formatted at 2026-08-09 16:20:46 on dist-test-slave-s11t"
server_key: "b1b6ae93f0803893fd6c4cf0553d50bd"
server_key_iv: "b17d0d98dde063f81b0f917ad28a3e22"
server_key_version: "encryptionkey@0"
I20260809 16:20:46.121021 30959 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260809 16:20:46.121783 30982 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:20:46.121990 30959 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:20:46.122098 30959 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786292417722232-21710-0/minicluster-data/ts-2/data,/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786292417722232-21710-0/minicluster-data/ts-2/wal
uuid: "7c41c4e4decc4678b69c8506f0d7ad44"
format_stamp: "Formatted at 2026-08-09 16:20:46 on dist-test-slave-s11t"
server_key: "b1b6ae93f0803893fd6c4cf0553d50bd"
server_key_iv: "b17d0d98dde063f81b0f917ad28a3e22"
server_key_version: "encryptionkey@0"
I20260809 16:20:46.122174 30959 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786292417722232-21710-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786292417722232-21710-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786292417722232-21710-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:20:46.136224 30959 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:20:46.138845 30959 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:20:46.138969 30959 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:20:46.139137 30959 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:20:46.139381 30959 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:20:46.139412 30959 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:20:46.139434 30959 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:20:46.139447 30959 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:20:46.145597 30959 rpc_server.cc:307] RPC server started. Bound to: 127.21.51.131:43689
I20260809 16:20:46.145642 31151 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.51.131:43689 every 8 connection(s)
I20260809 16:20:46.145970 30959 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786292417722232-21710-0/minicluster-data/ts-2/data/info.pb
Aug 09 16:20:46 dist-test-slave-s11t krb5kdc[30364](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292446, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.51.131@KRBTEST.COM for kudu/127.21.51.190@KRBTEST.COM
I20260809 16:20:46.154839 21710 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrNqUCn/build/release/bin/kudu as pid 30959
I20260809 16:20:46.154924 21710 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786292417722232-21710-0/minicluster-data/ts-2/wal/instance
I20260809 16:20:46.155135 21710 external_mini_cluster.cc:1428] Setting key 9b9c84b9daaa12b9d74666da7f177a97
I20260809 16:20:46.155201 30492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:46.146678 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.51.131:43595 (local address 127.21.51.190:45603)
0809 16:20:46.146727 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0809 16:20:46.146728 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 16:20:46.146943 (+ 215us) server_negotiation.cc:407] Connection header received
0809 16:20:46.147850 (+ 907us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:20:46.147851 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:20:46.147861 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:20:46.147920 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:20:46.148493 (+ 573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:46.149054 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:46.150023 (+ 969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:46.150205 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:46.150256 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:46.152369 (+ 2113us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:20:46.152370 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:20:46.152371 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:20:46.152377 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:20:46.153655 (+ 1278us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:20:46.154031 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:20:46.154031 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:20:46.154032 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:20:46.154082 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:20:46.154490 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:20:46.154491 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:20:46.154492 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:20:46.154607 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:20:46.154679 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0809 16:20:46.155013 (+ 334us) server_negotiation.cc:299] Negotiation successful
0809 16:20:46.155103 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260809 16:20:46.155478 31154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:46.146640 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:45603 (local address 127.21.51.131:43595)
0809 16:20:46.146817 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0809 16:20:46.146859 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0809 16:20:46.147764 (+ 905us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:20:46.147981 (+ 217us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:20:46.147984 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:20:46.148229 (+ 245us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:20:46.148418 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:20:46.148422 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:20:46.149133 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:20:46.149134 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:20:46.149948 (+ 814us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:20:46.149950 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:20:46.150275 (+ 325us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:20:46.150276 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:20:46.150370 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:46.150866 (+ 496us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:20:46.150874 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:20:46.152275 (+ 1401us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:20:46.153726 (+ 1451us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:20:46.153732 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:20:46.153733 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:20:46.153964 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:20:46.154295 (+ 331us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:20:46.154296 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:20:46.154296 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:20:46.154410 (+ 114us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:20:46.154702 (+ 292us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:20:46.154708 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:20:46.154844 (+ 136us) client_negotiation.cc:769] Sending connection context
0809 16:20:46.154963 (+ 119us) client_negotiation.cc:240] Negotiation successful
0809 16:20:46.155120 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":99,"threads_started":1}
I20260809 16:20:46.155916 31152 heartbeater.cc:347] Connected to a master server at 127.21.51.190:45603
I20260809 16:20:46.156029 31152 heartbeater.cc:464] Registering TS with master...
I20260809 16:20:46.156205 31152 heartbeater.cc:511] Master 127.21.51.190:45603 requested a full tablet report, sending...
I20260809 16:20:46.156618 30435 ts_manager.cc:194] Registered new tserver with Master: 7c41c4e4decc4678b69c8506f0d7ad44 (127.21.51.131:43689)
I20260809 16:20:46.157017 30435 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.51.131@KRBTEST.COM'} at 127.21.51.131:43595
I20260809 16:20:46.167109 21710 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786292417722232-21710-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786292417722232-21710-0/krb5kdc/test-admin.keytab.
Aug 09 16:20:46 dist-test-slave-s11t krb5kdc[30364](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292446, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:20:46.184449 21710 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260809 16:20:46.186242 31162 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:20:46.185404 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39746 (local address 127.21.51.129:41679)
0809 16:20:46.185590 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0809 16:20:46.185595 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 16:20:46.185605 (+ 10us) server_negotiation.cc:407] Connection header received
0809 16:20:46.185630 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:20:46.185632 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:20:46.185645 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:20:46.185710 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:20:46.186140 (+ 430us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39746: BlockingRecv error: recv got EOF from 127.0.0.1:39746 (error 108)
Metrics: {"server-negotiator.queue_time_us":154,"thread_start_us":106,"threads_started":1}
W20260809 16:20:46.186331 21710 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786292417722232-21710-0/krb5kdc/krb5cc' of type FILE
Aug 09 16:20:46 dist-test-slave-s11t krb5kdc[30364](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292446, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:20:46.187302 21710 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 09 16:20:46 dist-test-slave-s11t krb5kdc[30364](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292446, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:20:46.188166 21710 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 09 16:20:46 dist-test-slave-s11t krb5kdc[30364](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292446, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.51.129@KRBTEST.COM
I20260809 16:20:46.195497 31162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:46.188710 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39750 (local address 127.21.51.129:41679)
0809 16:20:46.188775 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0809 16:20:46.188776 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 16:20:46.188799 (+ 23us) server_negotiation.cc:407] Connection header received
0809 16:20:46.188894 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:20:46.188894 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:20:46.188904 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:20:46.188957 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:20:46.189338 (+ 381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:46.190051 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:46.190735 (+ 684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:46.191038 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:46.191090 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:46.193146 (+ 2056us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:20:46.193153 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:20:46.193153 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:20:46.193159 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:20:46.194552 (+ 1393us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:20:46.194861 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:20:46.194864 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:20:46.194864 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:20:46.194916 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:20:46.195078 (+ 162us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:20:46.195079 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:20:46.195079 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:20:46.195197 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:20:46.195287 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0809 16:20:46.195368 (+ 81us) server_negotiation.cc:299] Negotiation successful
0809 16:20:46.195428 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
W20260809 16:20:46.197315 31162 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:20:46.196718 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39766 (local address 127.21.51.129:41679)
0809 16:20:46.196758 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0809 16:20:46.196759 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 16:20:46.196810 (+ 51us) server_negotiation.cc:407] Connection header received
0809 16:20:46.196913 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:20:46.196913 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:20:46.196924 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:20:46.196994 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:20:46.197264 (+ 270us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39766: BlockingRecv error: recv got EOF from 127.0.0.1:39766 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260809 16:20:46.197520 21710 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786292417722232-21710-0/krb5kdc/krb5cc' of type FILE
Aug 09 16:20:46 dist-test-slave-s11t krb5kdc[30364](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292446, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:20:46.198468 21710 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 09 16:20:46 dist-test-slave-s11t krb5kdc[30364](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292446, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:20:46.199321 21710 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 09 16:20:46 dist-test-slave-s11t krb5kdc[30364](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292446, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.51.129@KRBTEST.COM
I20260809 16:20:46.206152 31162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:46.199788 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39770 (local address 127.21.51.129:41679)
0809 16:20:46.199844 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0809 16:20:46.199845 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 16:20:46.199889 (+ 44us) server_negotiation.cc:407] Connection header received
0809 16:20:46.199925 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:20:46.199926 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:20:46.199933 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:20:46.199985 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:20:46.200316 (+ 331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:46.200846 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:46.201469 (+ 623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:46.201656 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:46.201705 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:46.203677 (+ 1972us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:20:46.203678 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:20:46.203678 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:20:46.203684 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:20:46.205092 (+ 1408us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:20:46.205428 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:20:46.205429 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:20:46.205429 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:20:46.205482 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:20:46.205693 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:20:46.205693 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:20:46.205694 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:20:46.205781 (+ 87us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:20:46.205852 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0809 16:20:46.205953 (+ 101us) server_negotiation.cc:299] Negotiation successful
0809 16:20:46.206014 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
W20260809 16:20:46.207937 31162 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:20:46.207329 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39780 (local address 127.21.51.129:41679)
0809 16:20:46.207392 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0809 16:20:46.207394 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:20:46.207406 (+ 12us) server_negotiation.cc:407] Connection header received
0809 16:20:46.207483 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:20:46.207484 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:20:46.207490 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:20:46.207538 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:20:46.207889 (+ 351us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39780: BlockingRecv error: recv got EOF from 127.0.0.1:39780 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
W20260809 16:20:46.208232 21710 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786292417722232-21710-0/krb5kdc/krb5cc' of type FILE
Aug 09 16:20:46 dist-test-slave-s11t krb5kdc[30364](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292446, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:20:46.209422 21710 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 09 16:20:46 dist-test-slave-s11t krb5kdc[30364](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292446, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:20:46.210402 21710 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 09 16:20:46 dist-test-slave-s11t krb5kdc[30364](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292446, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.51.129@KRBTEST.COM
I20260809 16:20:46.217003 31162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:46.210932 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39796 (local address 127.21.51.129:41679)
0809 16:20:46.210983 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0809 16:20:46.210984 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 16:20:46.211077 (+ 93us) server_negotiation.cc:407] Connection header received
0809 16:20:46.211126 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:20:46.211126 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:20:46.211134 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:20:46.211183 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:20:46.211535 (+ 352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:46.211970 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:46.212656 (+ 686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:46.212891 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:46.212943 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:46.214867 (+ 1924us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:20:46.214869 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:20:46.214869 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:20:46.214875 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:20:46.216046 (+ 1171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:20:46.216346 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:20:46.216347 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:20:46.216347 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:20:46.216400 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:20:46.216562 (+ 162us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:20:46.216563 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:20:46.216563 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:20:46.216646 (+ 83us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:20:46.216804 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0809 16:20:46.216895 (+ 91us) server_negotiation.cc:299] Negotiation successful
0809 16:20:46.216951 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260809 16:20:46.217628 21710 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrNqUCn/build/release/bin/kudu with pid 30533
I20260809 16:20:46.241092 21710 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrNqUCn/build/release/bin/kudu with pid 30734
I20260809 16:20:46.246492 21710 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrNqUCn/build/release/bin/kudu with pid 30959
I20260809 16:20:46.252041 21710 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrNqUCn/build/release/bin/kudu with pid 30383
2026-08-09T16:20:46Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (2114 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292417722232-21710-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 16:20:46 dist-test-slave-s11t krb5kdc[31184](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 16:20:46 dist-test-slave-s11t krb5kdc[31184](info): set up 2 sockets
Aug 09 16:20:46 dist-test-slave-s11t krb5kdc[31184](info): commencing operation
krb5kdc: starting...
W20260809 16:20:47.492439 21710 mini_kdc.cc:121] Time spent starting KDC: real 1.216s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292417722232-21710-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292417722232-21710-0/krb5kdc/test-user.keytab.
Aug 09 16:20:47 dist-test-slave-s11t krb5kdc[31184](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292447, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T16:20:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T16:20:47Z Disabled control of system clock
WARNING: no policy specified for oryx/127.21.51.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.21.51.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.21.51.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292417722232-21710-0/krb5kdc/oryx_127.21.51.190.keytab.
Entry for principal oryx/127.21.51.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292417722232-21710-0/krb5kdc/oryx_127.21.51.190.keytab.
I20260809 16:20:47.611663 21710 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrNqUCn/build/release/bin/kudu
/tmp/dist-test-taskrNqUCn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292417722232-21710-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292417722232-21710-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292417722232-21710-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292417722232-21710-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.51.190:40451
--webserver_interface=127.21.51.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292417722232-21710-0/krb5kdc/oryx_127.21.51.190.keytab
--principal=oryx/127.21.51.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.51.148:33645
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.51.190:40451
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292417722232-21710-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292417722232-21710-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292417722232-21710-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:20:47.694653 31203 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:20:47.694855 31203 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:20:47.694883 31203 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:20:47.695945 31203 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 16:20:47.695976 31203 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:20:47.695991 31203 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:20:47.696008 31203 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 16:20:47.696022 31203 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 16:20:47.697412 31203 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.51.148:33645
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292417722232-21710-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292417722232-21710-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.51.190:40451
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292417722232-21710-0/krb5kdc/oryx_127.21.51.190.keytab
--principal=oryx/127.21.51.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.51.190:40451
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292417722232-21710-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.51.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31203
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292417722232-21710-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:20:47.697599 31203 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:20:47.697767 31203 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:20:47.702826 31213 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:20:47.702831 31214 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:20:47.703539 31216 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:20:47.704595 31203 server_base.cc:1034] running on GCE node
I20260809 16:20:47.704866 31203 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:20:47.705116 31203 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:20:47.706254 31203 hybrid_clock.cc:648] HybridClock initialized: now 1786292447706242 us; error 52 us; skew 500 ppm
Aug 09 16:20:47 dist-test-slave-s11t krb5kdc[31184](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292447, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.51.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:20:47.708912 31203 init.cc:377] Logged in from keytab as oryx/127.21.51.190@KRBTEST.COM (short username oryx)
I20260809 16:20:47.709419 31203 webserver.cc:460] Webserver started at http://127.21.51.190:37239/ using document root <none> and password file <none>
I20260809 16:20:47.709649 31203 fs_manager.cc:359] Metadata directory not provided
I20260809 16:20:47.709719 31203 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:20:47.709874 31203 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:20:47.710934 31203 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292417722232-21710-0/minicluster-data/master-0/data/instance:
uuid: "4896338a36d9400f84bedb07f140d7ad"
format_stamp: "Formatted at 2026-08-09 16:20:47 on dist-test-slave-s11t"
server_key: "1f524604884a3255eb5ae622f9fee598"
server_key_iv: "f73a50451094a005832853cd64a03d88"
server_key_version: "encryptionkey@0"
I20260809 16:20:47.711303 31203 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292417722232-21710-0/minicluster-data/master-0/wal/instance:
uuid: "4896338a36d9400f84bedb07f140d7ad"
format_stamp: "Formatted at 2026-08-09 16:20:47 on dist-test-slave-s11t"
server_key: "1f524604884a3255eb5ae622f9fee598"
server_key_iv: "f73a50451094a005832853cd64a03d88"
server_key_version: "encryptionkey@0"
I20260809 16:20:47.712631 31203 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260809 16:20:47.713409 31226 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:20:47.713582 31203 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 16:20:47.713687 31203 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292417722232-21710-0/minicluster-data/master-0/data,/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292417722232-21710-0/minicluster-data/master-0/wal
uuid: "4896338a36d9400f84bedb07f140d7ad"
format_stamp: "Formatted at 2026-08-09 16:20:47 on dist-test-slave-s11t"
server_key: "1f524604884a3255eb5ae622f9fee598"
server_key_iv: "f73a50451094a005832853cd64a03d88"
server_key_version: "encryptionkey@0"
I20260809 16:20:47.713771 31203 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292417722232-21710-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292417722232-21710-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292417722232-21710-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:20:47.734540 31203 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:20:47.736295 31203 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:20:47.736472 31203 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:20:47.740375 31203 rpc_server.cc:307] RPC server started. Bound to: 127.21.51.190:40451
I20260809 16:20:47.740439 31317 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.51.190:40451 every 8 connection(s)
I20260809 16:20:47.740825 31203 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292417722232-21710-0/minicluster-data/master-0/data/info.pb
I20260809 16:20:47.741813 31318 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:20:47.744098 31318 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4896338a36d9400f84bedb07f140d7ad: Bootstrap starting.
I20260809 16:20:47.744721 31318 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4896338a36d9400f84bedb07f140d7ad: Neither blocks nor log segments found. Creating new log.
I20260809 16:20:47.745090 31318 log.cc:824] T 00000000000000000000000000000000 P 4896338a36d9400f84bedb07f140d7ad: Log is configured to *not* fsync() on all Append() calls
I20260809 16:20:47.745823 31318 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4896338a36d9400f84bedb07f140d7ad: No bootstrap required, opened a new log
I20260809 16:20:47.747107 21710 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrNqUCn/build/release/bin/kudu as pid 31203
I20260809 16:20:47.747225 21710 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292417722232-21710-0/minicluster-data/master-0/wal/instance
I20260809 16:20:47.747421 21710 external_mini_cluster.cc:1428] Setting key 35786c2ea260187fc170cc08d3d4cfb2
I20260809 16:20:47.747525 31318 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4896338a36d9400f84bedb07f140d7ad [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4896338a36d9400f84bedb07f140d7ad" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 40451 } }
I20260809 16:20:47.747699 31318 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4896338a36d9400f84bedb07f140d7ad [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:20:47.747740 31318 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4896338a36d9400f84bedb07f140d7ad [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4896338a36d9400f84bedb07f140d7ad, State: Initialized, Role: FOLLOWER
I20260809 16:20:47.747922 31318 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4896338a36d9400f84bedb07f140d7ad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4896338a36d9400f84bedb07f140d7ad" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 40451 } }
I20260809 16:20:47.748025 31318 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4896338a36d9400f84bedb07f140d7ad [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 16:20:47.748062 31318 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4896338a36d9400f84bedb07f140d7ad [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 16:20:47.748112 31318 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4896338a36d9400f84bedb07f140d7ad [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:20:47.749076 31318 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4896338a36d9400f84bedb07f140d7ad [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4896338a36d9400f84bedb07f140d7ad" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 40451 } }
I20260809 16:20:47.749202 31318 leader_election.cc:302] T 00000000000000000000000000000000 P 4896338a36d9400f84bedb07f140d7ad [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4896338a36d9400f84bedb07f140d7ad; no voters:
I20260809 16:20:47.749361 31318 leader_election.cc:288] T 00000000000000000000000000000000 P 4896338a36d9400f84bedb07f140d7ad [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 16:20:47.749403 31327 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4896338a36d9400f84bedb07f140d7ad [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:20:47.749595 31327 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4896338a36d9400f84bedb07f140d7ad [term 1 LEADER]: Becoming Leader. State: Replica: 4896338a36d9400f84bedb07f140d7ad, State: Running, Role: LEADER
I20260809 16:20:47.749647 31318 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4896338a36d9400f84bedb07f140d7ad [sys.catalog]: configured and running, proceeding with master startup.
I20260809 16:20:47.749796 31327 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4896338a36d9400f84bedb07f140d7ad [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4896338a36d9400f84bedb07f140d7ad" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 40451 } }
I20260809 16:20:47.750239 31329 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4896338a36d9400f84bedb07f140d7ad [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4896338a36d9400f84bedb07f140d7ad. Latest consensus state: current_term: 1 leader_uuid: "4896338a36d9400f84bedb07f140d7ad" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4896338a36d9400f84bedb07f140d7ad" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 40451 } } }
I20260809 16:20:47.750451 31329 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4896338a36d9400f84bedb07f140d7ad [sys.catalog]: This master's current role is: LEADER
I20260809 16:20:47.750396 31328 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4896338a36d9400f84bedb07f140d7ad [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4896338a36d9400f84bedb07f140d7ad" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4896338a36d9400f84bedb07f140d7ad" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 40451 } } }
I20260809 16:20:47.750706 31328 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4896338a36d9400f84bedb07f140d7ad [sys.catalog]: This master's current role is: LEADER
I20260809 16:20:47.750885 31339 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 16:20:47.751434 31339 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 16:20:47.753050 31339 catalog_manager.cc:1347] Generated new cluster ID: 70b9d0b7af95488b90d1785aca0f4ff6
I20260809 16:20:47.753103 31339 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 09 16:20:47 dist-test-slave-s11t krb5kdc[31184](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292447, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.21.51.190@KRBTEST.COM
I20260809 16:20:47.759065 31325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:47.747926 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40086 (local address 127.21.51.190:40451)
0809 16:20:47.748202 (+ 276us) server_negotiation.cc:206] Beginning negotiation
0809 16:20:47.748219 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0809 16:20:47.748247 (+ 28us) server_negotiation.cc:407] Connection header received
0809 16:20:47.748343 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:20:47.748348 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:20:47.748406 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:20:47.748539 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:20:47.749245 (+ 706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:47.750600 (+ 1355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:47.751665 (+ 1065us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:47.752258 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:47.752386 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:47.755223 (+ 2837us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:20:47.755228 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:20:47.755228 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:20:47.755240 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:20:47.757215 (+ 1975us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:20:47.757635 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:20:47.757638 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:20:47.757639 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:20:47.757962 (+ 323us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:20:47.758201 (+ 239us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:20:47.758202 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:20:47.758202 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:20:47.758481 (+ 279us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:20:47.758623 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0809 16:20:47.758752 (+ 129us) server_negotiation.cc:299] Negotiation successful
0809 16:20:47.758882 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":222,"thread_start_us":79,"threads_started":1}
I20260809 16:20:47.771605 31339 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 16:20:47.772324 31339 catalog_manager.cc:1504] Loading token signing keys...
I20260809 16:20:47.783048 31339 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4896338a36d9400f84bedb07f140d7ad: Generated new TSK 0
I20260809 16:20:47.783231 31339 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.21.51.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.21.51.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.21.51.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292417722232-21710-0/krb5kdc/oryx_127.21.51.129.keytab.
Entry for principal oryx/127.21.51.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292417722232-21710-0/krb5kdc/oryx_127.21.51.129.keytab.
I20260809 16:20:47.813534 21710 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrNqUCn/build/release/bin/kudu
/tmp/dist-test-taskrNqUCn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292417722232-21710-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292417722232-21710-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292417722232-21710-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292417722232-21710-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.51.129:0
--local_ip_for_outbound_sockets=127.21.51.129
--webserver_interface=127.21.51.129
--webserver_port=0
--tserver_master_addrs=127.21.51.190:40451
--keytab_file=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292417722232-21710-0/krb5kdc/oryx_127.21.51.129.keytab
--principal=oryx/127.21.51.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.51.148:33645
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292417722232-21710-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292417722232-21710-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292417722232-21710-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:20:47.895378 31354 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:20:47.895570 31354 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:20:47.895592 31354 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:20:47.896756 31354 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:20:47.896797 31354 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:20:47.896823 31354 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.51.129
I20260809 16:20:47.898379 31354 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.51.148:33645
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292417722232-21710-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292417722232-21710-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292417722232-21710-0/krb5kdc/oryx_127.21.51.129.keytab
--principal=oryx/127.21.51.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.51.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292417722232-21710-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.51.129
--webserver_port=0
--tserver_master_addrs=127.21.51.190:40451
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31354
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.51.129
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292417722232-21710-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:20:47.898634 31354 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:20:47.898882 31354 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:20:47.903976 31368 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:20:47.904016 31364 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:20:47.903982 31360 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:20:47.904227 31354 server_base.cc:1034] running on GCE node
I20260809 16:20:47.904400 31354 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:20:47.904583 31354 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:20:47.905723 31354 hybrid_clock.cc:648] HybridClock initialized: now 1786292447905716 us; error 43 us; skew 500 ppm
Aug 09 16:20:47 dist-test-slave-s11t krb5kdc[31184](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292447, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.51.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:20:47.908254 31354 init.cc:377] Logged in from keytab as oryx/127.21.51.129@KRBTEST.COM (short username oryx)
I20260809 16:20:47.908792 31354 webserver.cc:460] Webserver started at http://127.21.51.129:39855/ using document root <none> and password file <none>
I20260809 16:20:47.909039 31354 fs_manager.cc:359] Metadata directory not provided
I20260809 16:20:47.909109 31354 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:20:47.909248 31354 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:20:47.910259 31354 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292417722232-21710-0/minicluster-data/ts-0/data/instance:
uuid: "62e7e0d86ddb442abe06e05b9bce2806"
format_stamp: "Formatted at 2026-08-09 16:20:47 on dist-test-slave-s11t"
server_key: "c11050fed4aa359d9f1ec10c20eb7dde"
server_key_iv: "875c836c1829390054293fa15abcc95e"
server_key_version: "encryptionkey@0"
I20260809 16:20:47.910629 31354 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292417722232-21710-0/minicluster-data/ts-0/wal/instance:
uuid: "62e7e0d86ddb442abe06e05b9bce2806"
format_stamp: "Formatted at 2026-08-09 16:20:47 on dist-test-slave-s11t"
server_key: "c11050fed4aa359d9f1ec10c20eb7dde"
server_key_iv: "875c836c1829390054293fa15abcc95e"
server_key_version: "encryptionkey@0"
I20260809 16:20:47.911959 31354 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 16:20:47.912729 31379 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:20:47.912940 31354 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 16:20:47.913064 31354 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292417722232-21710-0/minicluster-data/ts-0/data,/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292417722232-21710-0/minicluster-data/ts-0/wal
uuid: "62e7e0d86ddb442abe06e05b9bce2806"
format_stamp: "Formatted at 2026-08-09 16:20:47 on dist-test-slave-s11t"
server_key: "c11050fed4aa359d9f1ec10c20eb7dde"
server_key_iv: "875c836c1829390054293fa15abcc95e"
server_key_version: "encryptionkey@0"
I20260809 16:20:47.913156 31354 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292417722232-21710-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292417722232-21710-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292417722232-21710-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:20:47.939275 31354 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:20:47.941288 31354 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:20:47.941468 31354 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:20:47.941716 31354 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:20:47.942044 31354 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:20:47.942098 31354 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:20:47.942140 31354 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:20:47.942171 31354 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:20:47.948091 31354 rpc_server.cc:307] RPC server started. Bound to: 127.21.51.129:33587
I20260809 16:20:47.948136 31554 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.51.129:33587 every 8 connection(s)
I20260809 16:20:47.948465 31354 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292417722232-21710-0/minicluster-data/ts-0/data/info.pb
Aug 09 16:20:47 dist-test-slave-s11t krb5kdc[31184](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292447, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.51.129@KRBTEST.COM for oryx/127.21.51.190@KRBTEST.COM
I20260809 16:20:47.957872 31325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:47.949206 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.51.129:53735 (local address 127.21.51.190:40451)
0809 16:20:47.949279 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0809 16:20:47.949281 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:20:47.949467 (+ 186us) server_negotiation.cc:407] Connection header received
0809 16:20:47.949991 (+ 524us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:20:47.949992 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:20:47.950007 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:20:47.950067 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:20:47.951184 (+ 1117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:47.951806 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:47.952792 (+ 986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:47.953003 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:47.953080 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:47.955178 (+ 2098us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:20:47.955179 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:20:47.955180 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:20:47.955186 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:20:47.956446 (+ 1260us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:20:47.957000 (+ 554us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:20:47.957001 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:20:47.957002 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:20:47.957063 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:20:47.957318 (+ 255us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:20:47.957319 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:20:47.957319 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:20:47.957424 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:20:47.957479 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0809 16:20:47.957731 (+ 252us) server_negotiation.cc:299] Negotiation successful
0809 16:20:47.957810 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260809 16:20:47.958031 31557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:47.949131 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:40451 (local address 127.21.51.129:53735)
0809 16:20:47.949336 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0809 16:20:47.949378 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0809 16:20:47.949894 (+ 516us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:20:47.950147 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:20:47.950150 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:20:47.950865 (+ 715us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:20:47.951064 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:20:47.951069 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:20:47.951888 (+ 819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:20:47.951889 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:20:47.952672 (+ 783us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:20:47.952673 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:20:47.953099 (+ 426us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:20:47.953100 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:20:47.953230 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:47.953659 (+ 429us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:20:47.953666 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:20:47.955093 (+ 1427us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:20:47.956547 (+ 1454us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:20:47.956554 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:20:47.956556 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:20:47.956908 (+ 352us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:20:47.957128 (+ 220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:20:47.957128 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:20:47.957129 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:20:47.957237 (+ 108us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:20:47.957484 (+ 247us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:20:47.957489 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:20:47.957606 (+ 117us) client_negotiation.cc:769] Sending connection context
0809 16:20:47.957694 (+ 88us) client_negotiation.cc:240] Negotiation successful
0809 16:20:47.957822 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":94,"threads_started":1}
I20260809 16:20:47.958547 21710 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrNqUCn/build/release/bin/kudu as pid 31354
I20260809 16:20:47.958659 21710 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292417722232-21710-0/minicluster-data/ts-0/wal/instance
I20260809 16:20:47.958654 31555 heartbeater.cc:347] Connected to a master server at 127.21.51.190:40451
I20260809 16:20:47.958750 31555 heartbeater.cc:464] Registering TS with master...
I20260809 16:20:47.958899 21710 external_mini_cluster.cc:1428] Setting key eb3a7ad4fe801fb7b534eb260ac157f4
I20260809 16:20:47.958951 31555 heartbeater.cc:511] Master 127.21.51.190:40451 requested a full tablet report, sending...
I20260809 16:20:47.959573 31258 ts_manager.cc:194] Registered new tserver with Master: 62e7e0d86ddb442abe06e05b9bce2806 (127.21.51.129:33587)
I20260809 16:20:47.960203 31258 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.21.51.129@KRBTEST.COM'} at 127.21.51.129:53735
WARNING: no policy specified for oryx/127.21.51.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.21.51.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.21.51.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292417722232-21710-0/krb5kdc/oryx_127.21.51.130.keytab.
Entry for principal oryx/127.21.51.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292417722232-21710-0/krb5kdc/oryx_127.21.51.130.keytab.
I20260809 16:20:47.987530 21710 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrNqUCn/build/release/bin/kudu
/tmp/dist-test-taskrNqUCn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292417722232-21710-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292417722232-21710-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292417722232-21710-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292417722232-21710-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.51.130:0
--local_ip_for_outbound_sockets=127.21.51.130
--webserver_interface=127.21.51.130
--webserver_port=0
--tserver_master_addrs=127.21.51.190:40451
--keytab_file=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292417722232-21710-0/krb5kdc/oryx_127.21.51.130.keytab
--principal=oryx/127.21.51.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.51.148:33645
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292417722232-21710-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292417722232-21710-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292417722232-21710-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:20:48.069479 31561 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:20:48.069686 31561 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:20:48.069710 31561 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:20:48.070772 31561 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:20:48.070806 31561 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:20:48.070830 31561 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.51.130
I20260809 16:20:48.072217 31561 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.51.148:33645
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292417722232-21710-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292417722232-21710-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292417722232-21710-0/krb5kdc/oryx_127.21.51.130.keytab
--principal=oryx/127.21.51.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.51.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292417722232-21710-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.51.130
--webserver_port=0
--tserver_master_addrs=127.21.51.190:40451
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31561
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.51.130
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292417722232-21710-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:20:48.072405 31561 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:20:48.072579 31561 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:20:48.077783 31575 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:20:48.077780 31571 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:20:48.077797 31572 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:20:48.077944 31561 server_base.cc:1034] running on GCE node
I20260809 16:20:48.078282 31561 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:20:48.078528 31561 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:20:48.079710 31561 hybrid_clock.cc:648] HybridClock initialized: now 1786292448079697 us; error 29 us; skew 500 ppm
Aug 09 16:20:48 dist-test-slave-s11t krb5kdc[31184](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292448, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.51.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:20:48.082443 31561 init.cc:377] Logged in from keytab as oryx/127.21.51.130@KRBTEST.COM (short username oryx)
I20260809 16:20:48.082981 31561 webserver.cc:460] Webserver started at http://127.21.51.130:35549/ using document root <none> and password file <none>
I20260809 16:20:48.083226 31561 fs_manager.cc:359] Metadata directory not provided
I20260809 16:20:48.083297 31561 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:20:48.083454 31561 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:20:48.084506 31561 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292417722232-21710-0/minicluster-data/ts-1/data/instance:
uuid: "d3af45dba9b04300879578698c6a2922"
format_stamp: "Formatted at 2026-08-09 16:20:48 on dist-test-slave-s11t"
server_key: "6da352be3b2a44ddc55026aaf1bdd162"
server_key_iv: "a7e2578e84826695c298e53cce96c8dc"
server_key_version: "encryptionkey@0"
I20260809 16:20:48.084914 31561 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292417722232-21710-0/minicluster-data/ts-1/wal/instance:
uuid: "d3af45dba9b04300879578698c6a2922"
format_stamp: "Formatted at 2026-08-09 16:20:48 on dist-test-slave-s11t"
server_key: "6da352be3b2a44ddc55026aaf1bdd162"
server_key_iv: "a7e2578e84826695c298e53cce96c8dc"
server_key_version: "encryptionkey@0"
I20260809 16:20:48.086315 31561 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260809 16:20:48.087100 31587 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:20:48.087270 31561 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 16:20:48.087375 31561 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292417722232-21710-0/minicluster-data/ts-1/data,/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292417722232-21710-0/minicluster-data/ts-1/wal
uuid: "d3af45dba9b04300879578698c6a2922"
format_stamp: "Formatted at 2026-08-09 16:20:48 on dist-test-slave-s11t"
server_key: "6da352be3b2a44ddc55026aaf1bdd162"
server_key_iv: "a7e2578e84826695c298e53cce96c8dc"
server_key_version: "encryptionkey@0"
I20260809 16:20:48.087461 31561 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292417722232-21710-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292417722232-21710-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292417722232-21710-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:20:48.104413 31561 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:20:48.106387 31561 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:20:48.106555 31561 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:20:48.106792 31561 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:20:48.107110 31561 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:20:48.107167 31561 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:20:48.107201 31561 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:20:48.107229 31561 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:20:48.113637 31561 rpc_server.cc:307] RPC server started. Bound to: 127.21.51.130:34781
I20260809 16:20:48.113687 31766 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.51.130:34781 every 8 connection(s)
I20260809 16:20:48.113996 31561 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292417722232-21710-0/minicluster-data/ts-1/data/info.pb
Aug 09 16:20:48 dist-test-slave-s11t krb5kdc[31184](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292448, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.51.130@KRBTEST.COM for oryx/127.21.51.190@KRBTEST.COM
I20260809 16:20:48.122617 21710 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrNqUCn/build/release/bin/kudu as pid 31561
I20260809 16:20:48.122754 21710 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292417722232-21710-0/minicluster-data/ts-1/wal/instance
I20260809 16:20:48.122911 31325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:48.114650 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.51.130:54755 (local address 127.21.51.190:40451)
0809 16:20:48.114724 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0809 16:20:48.114726 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:20:48.114930 (+ 204us) server_negotiation.cc:407] Connection header received
0809 16:20:48.115369 (+ 439us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:20:48.115370 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:20:48.115382 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:20:48.115446 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:20:48.116507 (+ 1061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:48.117170 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:48.118138 (+ 968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:48.118315 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:48.118365 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:48.120335 (+ 1970us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:20:48.120336 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:20:48.120337 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:20:48.120343 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:20:48.121667 (+ 1324us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:20:48.122035 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:20:48.122035 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:20:48.122036 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:20:48.122084 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:20:48.122290 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:20:48.122291 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:20:48.122291 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:20:48.122383 (+ 92us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:20:48.122439 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0809 16:20:48.122717 (+ 278us) server_negotiation.cc:299] Negotiation successful
0809 16:20:48.122812 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260809 16:20:48.122991 21710 external_mini_cluster.cc:1428] Setting key 4789789411006ef7ef7a0c80db97fb48
I20260809 16:20:48.123107 31769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:48.114591 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:40451 (local address 127.21.51.130:54755)
0809 16:20:48.114754 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0809 16:20:48.114795 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0809 16:20:48.115280 (+ 485us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:20:48.115495 (+ 215us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:20:48.115499 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:20:48.116229 (+ 730us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:20:48.116415 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:20:48.116420 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:20:48.117249 (+ 829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:20:48.117250 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:20:48.118059 (+ 809us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:20:48.118061 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:20:48.118385 (+ 324us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:20:48.118386 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:20:48.118487 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:48.118936 (+ 449us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:20:48.118944 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:20:48.120249 (+ 1305us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:20:48.121739 (+ 1490us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:20:48.121745 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:20:48.121746 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:20:48.121969 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:20:48.122142 (+ 173us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:20:48.122143 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:20:48.122143 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:20:48.122237 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:20:48.122460 (+ 223us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:20:48.122464 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:20:48.122575 (+ 111us) client_negotiation.cc:769] Sending connection context
0809 16:20:48.122654 (+ 79us) client_negotiation.cc:240] Negotiation successful
0809 16:20:48.122854 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":95,"threads_started":1}
I20260809 16:20:48.123632 31767 heartbeater.cc:347] Connected to a master server at 127.21.51.190:40451
I20260809 16:20:48.123752 31767 heartbeater.cc:464] Registering TS with master...
I20260809 16:20:48.123929 31767 heartbeater.cc:511] Master 127.21.51.190:40451 requested a full tablet report, sending...
I20260809 16:20:48.124333 31258 ts_manager.cc:194] Registered new tserver with Master: d3af45dba9b04300879578698c6a2922 (127.21.51.130:34781)
I20260809 16:20:48.124851 31258 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.21.51.130@KRBTEST.COM'} at 127.21.51.130:54755
WARNING: no policy specified for oryx/127.21.51.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.21.51.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.21.51.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292417722232-21710-0/krb5kdc/oryx_127.21.51.131.keytab.
Entry for principal oryx/127.21.51.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292417722232-21710-0/krb5kdc/oryx_127.21.51.131.keytab.
I20260809 16:20:48.148856 21710 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrNqUCn/build/release/bin/kudu
/tmp/dist-test-taskrNqUCn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292417722232-21710-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292417722232-21710-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292417722232-21710-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292417722232-21710-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.51.131:0
--local_ip_for_outbound_sockets=127.21.51.131
--webserver_interface=127.21.51.131
--webserver_port=0
--tserver_master_addrs=127.21.51.190:40451
--keytab_file=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292417722232-21710-0/krb5kdc/oryx_127.21.51.131.keytab
--principal=oryx/127.21.51.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.51.148:33645
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292417722232-21710-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292417722232-21710-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292417722232-21710-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:20:48.229802 31776 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:20:48.230020 31776 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:20:48.230057 31776 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:20:48.231191 31776 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:20:48.231263 31776 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:20:48.231307 31776 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.51.131
I20260809 16:20:48.232758 31776 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.51.148:33645
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292417722232-21710-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292417722232-21710-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292417722232-21710-0/krb5kdc/oryx_127.21.51.131.keytab
--principal=oryx/127.21.51.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.51.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292417722232-21710-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.51.131
--webserver_port=0
--tserver_master_addrs=127.21.51.190:40451
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31776
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.51.131
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292417722232-21710-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:20:48.233011 31776 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:20:48.233237 31776 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:20:48.238430 31783 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:20:48.238517 31776 server_base.cc:1034] running on GCE node
W20260809 16:20:48.238472 31788 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:20:48.238430 31785 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:20:48.238934 31776 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:20:48.239202 31776 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:20:48.240378 31776 hybrid_clock.cc:648] HybridClock initialized: now 1786292448240367 us; error 30 us; skew 500 ppm
Aug 09 16:20:48 dist-test-slave-s11t krb5kdc[31184](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292448, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.51.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:20:48.242949 31776 init.cc:377] Logged in from keytab as oryx/127.21.51.131@KRBTEST.COM (short username oryx)
I20260809 16:20:48.243391 31776 webserver.cc:460] Webserver started at http://127.21.51.131:33963/ using document root <none> and password file <none>
I20260809 16:20:48.243569 31776 fs_manager.cc:359] Metadata directory not provided
I20260809 16:20:48.243609 31776 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:20:48.243700 31776 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:20:48.244856 31776 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292417722232-21710-0/minicluster-data/ts-2/data/instance:
uuid: "2bf53ab985e048d19a823bea592a496d"
format_stamp: "Formatted at 2026-08-09 16:20:48 on dist-test-slave-s11t"
server_key: "bb0534026278ef00034db15d6b7da243"
server_key_iv: "7c749655fa21bf7b9c100a0407aeaec5"
server_key_version: "encryptionkey@0"
I20260809 16:20:48.245308 31776 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292417722232-21710-0/minicluster-data/ts-2/wal/instance:
uuid: "2bf53ab985e048d19a823bea592a496d"
format_stamp: "Formatted at 2026-08-09 16:20:48 on dist-test-slave-s11t"
server_key: "bb0534026278ef00034db15d6b7da243"
server_key_iv: "7c749655fa21bf7b9c100a0407aeaec5"
server_key_version: "encryptionkey@0"
I20260809 16:20:48.247354 31776 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20260809 16:20:48.248344 31801 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:20:48.248554 31776 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:20:48.248657 31776 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292417722232-21710-0/minicluster-data/ts-2/data,/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292417722232-21710-0/minicluster-data/ts-2/wal
uuid: "2bf53ab985e048d19a823bea592a496d"
format_stamp: "Formatted at 2026-08-09 16:20:48 on dist-test-slave-s11t"
server_key: "bb0534026278ef00034db15d6b7da243"
server_key_iv: "7c749655fa21bf7b9c100a0407aeaec5"
server_key_version: "encryptionkey@0"
I20260809 16:20:48.248751 31776 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292417722232-21710-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292417722232-21710-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292417722232-21710-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:20:48.262440 31776 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:20:48.265266 31776 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:20:48.265457 31776 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:20:48.265699 31776 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:20:48.266021 31776 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:20:48.266059 31776 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:20:48.266083 31776 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:20:48.266098 31776 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:20:48.272099 31776 rpc_server.cc:307] RPC server started. Bound to: 127.21.51.131:46809
I20260809 16:20:48.272162 31971 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.51.131:46809 every 8 connection(s)
I20260809 16:20:48.272399 31776 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292417722232-21710-0/minicluster-data/ts-2/data/info.pb
I20260809 16:20:48.273756 21710 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrNqUCn/build/release/bin/kudu as pid 31776
I20260809 16:20:48.273851 21710 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292417722232-21710-0/minicluster-data/ts-2/wal/instance
I20260809 16:20:48.274082 21710 external_mini_cluster.cc:1428] Setting key 912f1e284852c52a29679b7741578869
Aug 09 16:20:48 dist-test-slave-s11t krb5kdc[31184](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292448, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.51.131@KRBTEST.COM for oryx/127.21.51.190@KRBTEST.COM
I20260809 16:20:48.281524 31325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:48.273099 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.51.131:44791 (local address 127.21.51.190:40451)
0809 16:20:48.273163 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0809 16:20:48.273165 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:20:48.273333 (+ 168us) server_negotiation.cc:407] Connection header received
0809 16:20:48.273960 (+ 627us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:20:48.273961 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:20:48.273975 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:20:48.274055 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:20:48.275337 (+ 1282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:48.275833 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:48.276840 (+ 1007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:48.277032 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:48.277090 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:48.279126 (+ 2036us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:20:48.279128 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:20:48.279128 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:20:48.279135 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:20:48.280265 (+ 1130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:20:48.280629 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:20:48.280630 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:20:48.280630 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:20:48.280716 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:20:48.280935 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:20:48.280936 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:20:48.280936 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:20:48.281041 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:20:48.281100 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0809 16:20:48.281393 (+ 293us) server_negotiation.cc:299] Negotiation successful
0809 16:20:48.281459 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260809 16:20:48.281742 31976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:48.273049 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:40451 (local address 127.21.51.131:44791)
0809 16:20:48.273210 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0809 16:20:48.273250 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0809 16:20:48.273857 (+ 607us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:20:48.274124 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:20:48.274129 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:20:48.274956 (+ 827us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:20:48.275249 (+ 293us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:20:48.275261 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:20:48.275911 (+ 650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:20:48.275912 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:20:48.276757 (+ 845us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:20:48.276760 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:20:48.277105 (+ 345us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:20:48.277106 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:20:48.277204 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:48.277713 (+ 509us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:20:48.277721 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:20:48.279043 (+ 1322us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:20:48.280338 (+ 1295us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:20:48.280343 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:20:48.280345 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:20:48.280562 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:20:48.280775 (+ 213us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:20:48.280775 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:20:48.280776 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:20:48.280875 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:20:48.281133 (+ 258us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:20:48.281139 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:20:48.281262 (+ 123us) client_negotiation.cc:769] Sending connection context
0809 16:20:48.281360 (+ 98us) client_negotiation.cc:240] Negotiation successful
0809 16:20:48.281484 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":93,"threads_started":1}
I20260809 16:20:48.282248 31973 heartbeater.cc:347] Connected to a master server at 127.21.51.190:40451
I20260809 16:20:48.282349 31973 heartbeater.cc:464] Registering TS with master...
I20260809 16:20:48.282536 31973 heartbeater.cc:511] Master 127.21.51.190:40451 requested a full tablet report, sending...
I20260809 16:20:48.282943 31258 ts_manager.cc:194] Registered new tserver with Master: 2bf53ab985e048d19a823bea592a496d (127.21.51.131:46809)
I20260809 16:20:48.283380 31258 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.21.51.131@KRBTEST.COM'} at 127.21.51.131:44791
I20260809 16:20:48.285969 21710 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.21.51.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.21.51.189@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.21.51.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292417722232-21710-0/krb5kdc/oryx_127.21.51.189.keytab.
Entry for principal oryx/127.21.51.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292417722232-21710-0/krb5kdc/oryx_127.21.51.189.keytab.
I20260809 16:20:48.311517 21710 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrNqUCn/build/release/bin/kudu
/tmp/dist-test-taskrNqUCn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292417722232-21710-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292417722232-21710-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292417722232-21710-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292417722232-21710-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.51.189:41439
--webserver_interface=127.21.51.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292417722232-21710-0/krb5kdc/oryx_127.21.51.189.keytab
--principal=oryx/127.21.51.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.51.148:33645
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.51.190:40451,127.21.51.189:41439
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292417722232-21710-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292417722232-21710-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292417722232-21710-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:20:48.393712 31980 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:20:48.393904 31980 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:20:48.393926 31980 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:20:48.395097 31980 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 16:20:48.395140 31980 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:20:48.395155 31980 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:20:48.395169 31980 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 16:20:48.395221 31980 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 16:20:48.396777 31980 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.51.148:33645
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292417722232-21710-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292417722232-21710-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.21.51.190:40451,127.21.51.189:41439
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292417722232-21710-0/krb5kdc/oryx_127.21.51.189.keytab
--principal=oryx/127.21.51.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.51.189:41439
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292417722232-21710-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.21.51.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31980
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292417722232-21710-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
W20260809 16:20:48.396903 31980 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.21.51.190:40451,127.21.51.189:41439'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260809 16:20:48.397047 31980 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:20:48.397279 31980 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:20:48.402235 31990 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:20:48.402258 31987 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:20:48.402439 31980 server_base.cc:1034] running on GCE node
W20260809 16:20:48.402537 31988 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:20:48.402824 31980 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:20:48.403164 31980 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:20:48.404326 31980 hybrid_clock.cc:648] HybridClock initialized: now 1786292448404321 us; error 37 us; skew 500 ppm
Aug 09 16:20:48 dist-test-slave-s11t krb5kdc[31184](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292448, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.51.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:20:48.406836 31980 init.cc:377] Logged in from keytab as oryx/127.21.51.189@KRBTEST.COM (short username oryx)
I20260809 16:20:48.407265 31980 webserver.cc:460] Webserver started at http://127.21.51.189:41215/ using document root <none> and password file <none>
I20260809 16:20:48.407519 31980 fs_manager.cc:359] Metadata directory not provided
I20260809 16:20:48.407593 31980 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:20:48.407742 31980 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:20:48.408851 31980 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292417722232-21710-0/minicluster-data/master-1/data/instance:
uuid: "e438fd6b8e6645f7addce3b7676caca0"
format_stamp: "Formatted at 2026-08-09 16:20:48 on dist-test-slave-s11t"
server_key: "ad1fb6f1e5c20f8256f04d770743d2a4"
server_key_iv: "1c992898d34a3059b986f184db127a42"
server_key_version: "encryptionkey@0"
I20260809 16:20:48.409210 31980 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292417722232-21710-0/minicluster-data/master-1/wal/instance:
uuid: "e438fd6b8e6645f7addce3b7676caca0"
format_stamp: "Formatted at 2026-08-09 16:20:48 on dist-test-slave-s11t"
server_key: "ad1fb6f1e5c20f8256f04d770743d2a4"
server_key_iv: "1c992898d34a3059b986f184db127a42"
server_key_version: "encryptionkey@0"
I20260809 16:20:48.410548 31980 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260809 16:20:48.411304 32001 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:20:48.411499 31980 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 16:20:48.411602 31980 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292417722232-21710-0/minicluster-data/master-1/data,/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292417722232-21710-0/minicluster-data/master-1/wal
uuid: "e438fd6b8e6645f7addce3b7676caca0"
format_stamp: "Formatted at 2026-08-09 16:20:48 on dist-test-slave-s11t"
server_key: "ad1fb6f1e5c20f8256f04d770743d2a4"
server_key_iv: "1c992898d34a3059b986f184db127a42"
server_key_version: "encryptionkey@0"
I20260809 16:20:48.411677 31980 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292417722232-21710-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292417722232-21710-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292417722232-21710-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:20:48.427570 31980 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:20:48.429663 31980 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:20:48.429836 31980 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:20:48.434044 31980 rpc_server.cc:307] RPC server started. Bound to: 127.21.51.189:41439
I20260809 16:20:48.434079 32081 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.51.189:41439 every 8 connection(s)
I20260809 16:20:48.434437 31980 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292417722232-21710-0/minicluster-data/master-1/data/info.pb
I20260809 16:20:48.435469 32082 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:20:48.436203 21710 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrNqUCn/build/release/bin/kudu as pid 31980
I20260809 16:20:48.437418 32082 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.51.190" port: 40451 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 16:20:48 dist-test-slave-s11t krb5kdc[31184](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292448, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.51.189@KRBTEST.COM for oryx/127.21.51.190@KRBTEST.COM
Aug 09 16:20:48 dist-test-slave-s11t krb5kdc[31184](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292447, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.21.51.189@KRBTEST.COM
I20260809 16:20:48.447222 31325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:48.438313 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40094 (local address 127.21.51.190:40451)
0809 16:20:48.438367 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0809 16:20:48.438369 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:20:48.438505 (+ 136us) server_negotiation.cc:407] Connection header received
0809 16:20:48.438560 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:20:48.438561 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:20:48.438572 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:20:48.438641 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:20:48.439111 (+ 470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:48.439901 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:48.441213 (+ 1312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:48.441416 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:48.441501 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:48.443697 (+ 2196us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:20:48.443698 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:20:48.443699 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:20:48.443705 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:20:48.445339 (+ 1634us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:20:48.445929 (+ 590us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:20:48.445930 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:20:48.445930 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:20:48.445984 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:20:48.446428 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:20:48.446429 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:20:48.446430 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:20:48.446641 (+ 211us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:20:48.446722 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0809 16:20:48.447062 (+ 340us) server_negotiation.cc:299] Negotiation successful
0809 16:20:48.447128 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260809 16:20:48.447626 32085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:48.436942 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48360 (local address 127.21.51.189:41439)
0809 16:20:48.437130 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0809 16:20:48.437140 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 16:20:48.437187 (+ 47us) server_negotiation.cc:407] Connection header received
0809 16:20:48.437611 (+ 424us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:20:48.437618 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:20:48.437663 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:20:48.437812 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:20:48.438571 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:48.439871 (+ 1300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:48.440962 (+ 1091us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:48.441282 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:48.441402 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:48.444277 (+ 2875us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:20:48.444284 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:20:48.444285 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:20:48.444291 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:20:48.446359 (+ 2068us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:20:48.446731 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:20:48.446733 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:20:48.446734 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:20:48.446796 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:20:48.446993 (+ 197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:20:48.446994 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:20:48.446994 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:20:48.447142 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:20:48.447310 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0809 16:20:48.447394 (+ 84us) server_negotiation.cc:299] Negotiation successful
0809 16:20:48.447466 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":139,"thread_start_us":86,"threads_started":1}
I20260809 16:20:48.447631 32086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:48.438206 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:40451 (local address 127.0.0.1:40094)
0809 16:20:48.438382 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0809 16:20:48.438429 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0809 16:20:48.438508 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:20:48.438666 (+ 158us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:20:48.438668 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:20:48.438924 (+ 256us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:20:48.439009 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:20:48.439013 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:20:48.440024 (+ 1011us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:20:48.440025 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:20:48.441126 (+ 1101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:20:48.441129 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:20:48.441513 (+ 384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:20:48.441514 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:20:48.441588 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:48.442114 (+ 526us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:20:48.442122 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:20:48.443577 (+ 1455us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:20:48.445414 (+ 1837us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:20:48.445419 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:20:48.445421 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:20:48.445838 (+ 417us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:20:48.446053 (+ 215us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:20:48.446054 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:20:48.446054 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:20:48.446182 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:20:48.446745 (+ 563us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:20:48.446749 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:20:48.446908 (+ 159us) client_negotiation.cc:769] Sending connection context
0809 16:20:48.447021 (+ 113us) client_negotiation.cc:240] Negotiation successful
0809 16:20:48.447173 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":101,"threads_started":1}
I20260809 16:20:48.448228 32082 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.51.189" port: 41439 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 16:20:48 dist-test-slave-s11t krb5kdc[31184](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292448, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.51.189@KRBTEST.COM for oryx/127.21.51.189@KRBTEST.COM
I20260809 16:20:48.454350 32086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:48.448474 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.189:41439 (local address 127.0.0.1:48364)
0809 16:20:48.448536 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0809 16:20:48.448549 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 16:20:48.448599 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:20:48.448747 (+ 148us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:20:48.448748 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:20:48.448874 (+ 126us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:20:48.448912 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:20:48.448913 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:20:48.449523 (+ 610us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:20:48.449524 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:20:48.450123 (+ 599us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:20:48.450124 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:20:48.450477 (+ 353us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:20:48.450478 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:20:48.450539 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:48.450892 (+ 353us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:20:48.450895 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:20:48.452007 (+ 1112us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:20:48.453398 (+ 1391us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:20:48.453399 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:20:48.453400 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:20:48.453558 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:20:48.453750 (+ 192us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:20:48.453751 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:20:48.453751 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:20:48.453804 (+ 53us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:20:48.454039 (+ 235us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:20:48.454040 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:20:48.454075 (+ 35us) client_negotiation.cc:769] Sending connection context
0809 16:20:48.454154 (+ 79us) client_negotiation.cc:240] Negotiation successful
0809 16:20:48.454245 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31}
I20260809 16:20:48.454409 32085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:48.448505 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48364 (local address 127.21.51.189:41439)
0809 16:20:48.448552 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0809 16:20:48.448554 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:20:48.448585 (+ 31us) server_negotiation.cc:407] Connection header received
0809 16:20:48.448642 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:20:48.448643 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:20:48.448651 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:20:48.448723 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:20:48.448988 (+ 265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:48.449436 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:48.450196 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:48.450408 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:48.450481 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:48.452088 (+ 1607us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:20:48.452090 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:20:48.452090 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:20:48.452096 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:20:48.453308 (+ 1212us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:20:48.453627 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:20:48.453628 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:20:48.453629 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:20:48.453682 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:20:48.453874 (+ 192us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:20:48.453875 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:20:48.453876 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:20:48.453974 (+ 98us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:20:48.454047 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0809 16:20:48.454187 (+ 140us) server_negotiation.cc:299] Negotiation successful
0809 16:20:48.454295 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":45,"server-negotiator.queue_time_us":31}
I20260809 16:20:48.455547 32082 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e438fd6b8e6645f7addce3b7676caca0: Bootstrap starting.
I20260809 16:20:48.456373 32082 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e438fd6b8e6645f7addce3b7676caca0: Neither blocks nor log segments found. Creating new log.
I20260809 16:20:48.456651 32082 log.cc:824] T 00000000000000000000000000000000 P e438fd6b8e6645f7addce3b7676caca0: Log is configured to *not* fsync() on all Append() calls
I20260809 16:20:48.457515 32082 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e438fd6b8e6645f7addce3b7676caca0: No bootstrap required, opened a new log
I20260809 16:20:48.458931 32082 raft_consensus.cc:348] T 00000000000000000000000000000000 P e438fd6b8e6645f7addce3b7676caca0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4896338a36d9400f84bedb07f140d7ad" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 40451 } } peers { permanent_uuid: "e438fd6b8e6645f7addce3b7676caca0" member_type: VOTER last_known_addr { host: "127.21.51.189" port: 41439 } }
I20260809 16:20:48.459059 32082 raft_consensus.cc:374] T 00000000000000000000000000000000 P e438fd6b8e6645f7addce3b7676caca0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:20:48.459081 32082 raft_consensus.cc:729] T 00000000000000000000000000000000 P e438fd6b8e6645f7addce3b7676caca0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e438fd6b8e6645f7addce3b7676caca0, State: Initialized, Role: FOLLOWER
I20260809 16:20:48.459239 32082 consensus_queue.cc:260] T 00000000000000000000000000000000 P e438fd6b8e6645f7addce3b7676caca0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4896338a36d9400f84bedb07f140d7ad" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 40451 } } peers { permanent_uuid: "e438fd6b8e6645f7addce3b7676caca0" member_type: VOTER last_known_addr { host: "127.21.51.189" port: 41439 } }
I20260809 16:20:48.459489 32088 sys_catalog.cc:455] T 00000000000000000000000000000000 P e438fd6b8e6645f7addce3b7676caca0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4896338a36d9400f84bedb07f140d7ad" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 40451 } } peers { permanent_uuid: "e438fd6b8e6645f7addce3b7676caca0" member_type: VOTER last_known_addr { host: "127.21.51.189" port: 41439 } } }
I20260809 16:20:48.459575 32088 sys_catalog.cc:458] T 00000000000000000000000000000000 P e438fd6b8e6645f7addce3b7676caca0 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 16:20:48.459553 32082 sys_catalog.cc:564] T 00000000000000000000000000000000 P e438fd6b8e6645f7addce3b7676caca0 [sys.catalog]: configured and running, proceeding with master startup.
W20260809 16:20:48.461253 32107 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e438fd6b8e6645f7addce3b7676caca0: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 16:20:48.461326 32107 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 16:20:48.462960 31980 master_runner.cc:428] Detected that this master e438fd6b8e6645f7addce3b7676caca0 is joining an existing cluster
I20260809 16:20:48.463024 31980 master_runner.cc:432] Initiating AddMaster RPC to add 127.21.51.189:41439
I20260809 16:20:48.470589 32085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:48.464324 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48376 (local address 127.21.51.189:41439)
0809 16:20:48.464358 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0809 16:20:48.464360 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:20:48.464461 (+ 101us) server_negotiation.cc:407] Connection header received
0809 16:20:48.464547 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:20:48.464548 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:20:48.464555 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:20:48.464581 (+ 26us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:20:48.465002 (+ 421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:48.465468 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:48.466201 (+ 733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:48.466407 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:48.466512 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:48.467566 (+ 1054us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:20:48.467568 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:20:48.467568 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:20:48.467575 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:20:48.469331 (+ 1756us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:20:48.469712 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:20:48.469713 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:20:48.469714 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:20:48.469776 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:20:48.470004 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:20:48.470005 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:20:48.470006 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:20:48.470134 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:20:48.470206 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0809 16:20:48.470383 (+ 177us) server_negotiation.cc:299] Negotiation successful
0809 16:20:48.470494 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":44,"server-negotiator.queue_time_us":18}
I20260809 16:20:48.470826 32120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:48.464283 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.189:41439 (local address 127.0.0.1:48376)
0809 16:20:48.464401 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0809 16:20:48.464409 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0809 16:20:48.464481 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:20:48.464629 (+ 148us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:20:48.464630 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:20:48.464874 (+ 244us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:20:48.464924 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:20:48.464926 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:20:48.465555 (+ 629us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:20:48.465556 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:20:48.466117 (+ 561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:20:48.466118 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:20:48.466527 (+ 409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:20:48.466527 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:20:48.466599 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:48.467079 (+ 480us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:20:48.467084 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:20:48.467497 (+ 413us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:20:48.469416 (+ 1919us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:20:48.469416 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:20:48.469417 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:20:48.469627 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:20:48.469858 (+ 231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:20:48.469859 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:20:48.469859 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:20:48.469929 (+ 70us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:20:48.470206 (+ 277us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:20:48.470206 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:20:48.470269 (+ 63us) client_negotiation.cc:769] Sending connection context
0809 16:20:48.470342 (+ 73us) client_negotiation.cc:240] Negotiation successful
0809 16:20:48.470707 (+ 365us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":81,"threads_started":1}
I20260809 16:20:48.471416 32119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:48.464071 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:40451 (local address 127.0.0.1:40104)
0809 16:20:48.464262 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0809 16:20:48.464276 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 16:20:48.464384 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:20:48.464556 (+ 172us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:20:48.464556 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:20:48.464822 (+ 266us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:20:48.464921 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:20:48.464924 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:20:48.465886 (+ 962us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:20:48.465887 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:20:48.466794 (+ 907us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:20:48.466797 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:20:48.467253 (+ 456us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:20:48.467254 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:20:48.467332 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:48.467733 (+ 401us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:20:48.467737 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:20:48.468144 (+ 407us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:20:48.470135 (+ 1991us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:20:48.470135 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:20:48.470136 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:20:48.470358 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:20:48.470594 (+ 236us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:20:48.470595 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:20:48.470595 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:20:48.470682 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:20:48.471090 (+ 408us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:20:48.471091 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:20:48.471131 (+ 40us) client_negotiation.cc:769] Sending connection context
0809 16:20:48.471204 (+ 73us) client_negotiation.cc:240] Negotiation successful
0809 16:20:48.471314 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":88,"threads_started":1}
I20260809 16:20:48.471442 31325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:48.464171 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40104 (local address 127.21.51.190:40451)
0809 16:20:48.464223 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0809 16:20:48.464225 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:20:48.464351 (+ 126us) server_negotiation.cc:407] Connection header received
0809 16:20:48.464479 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:20:48.464481 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:20:48.464493 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:20:48.464550 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:20:48.465022 (+ 472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:48.465787 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:48.466882 (+ 1095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:48.467155 (+ 273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:48.467250 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:48.468237 (+ 987us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:20:48.468238 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:20:48.468239 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:20:48.468247 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:20:48.470042 (+ 1795us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:20:48.470456 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:20:48.470457 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:20:48.470457 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:20:48.470519 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:20:48.470765 (+ 246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:20:48.470766 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:20:48.470766 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:20:48.471005 (+ 239us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:20:48.471065 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0809 16:20:48.471258 (+ 193us) server_negotiation.cc:299] Negotiation successful
0809 16:20:48.471340 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
Aug 09 16:20:48 dist-test-slave-s11t krb5kdc[31184](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292447, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.51.190@KRBTEST.COM for oryx/127.21.51.189@KRBTEST.COM
I20260809 16:20:48.481148 32123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:48.473773 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.189:41439 (local address 127.0.0.1:48382)
0809 16:20:48.473920 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0809 16:20:48.473962 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0809 16:20:48.474034 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:20:48.474172 (+ 138us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:20:48.474173 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:20:48.474443 (+ 270us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:20:48.474496 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:20:48.474499 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:20:48.475457 (+ 958us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:20:48.475458 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:20:48.476040 (+ 582us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:20:48.476042 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:20:48.476531 (+ 489us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:20:48.476532 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:20:48.476600 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:48.477078 (+ 478us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:20:48.477083 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:20:48.478558 (+ 1475us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:20:48.480143 (+ 1585us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:20:48.480150 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:20:48.480152 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:20:48.480346 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:20:48.480551 (+ 205us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:20:48.480551 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:20:48.480552 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:20:48.480607 (+ 55us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:20:48.480873 (+ 266us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:20:48.480880 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:20:48.480924 (+ 44us) client_negotiation.cc:769] Sending connection context
0809 16:20:48.480993 (+ 69us) client_negotiation.cc:240] Negotiation successful
0809 16:20:48.481080 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":100,"threads_started":1}
I20260809 16:20:48.481202 32085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:48.473961 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48382 (local address 127.21.51.189:41439)
0809 16:20:48.474025 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0809 16:20:48.474027 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:20:48.474037 (+ 10us) server_negotiation.cc:407] Connection header received
0809 16:20:48.474090 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:20:48.474091 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:20:48.474102 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:20:48.474167 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:20:48.474571 (+ 404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:48.475275 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:48.476126 (+ 851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:48.476431 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:48.476499 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:48.478644 (+ 2145us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:20:48.478645 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:20:48.478645 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:20:48.478652 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:20:48.480062 (+ 1410us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:20:48.480423 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:20:48.480424 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:20:48.480425 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:20:48.480482 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:20:48.480668 (+ 186us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:20:48.480669 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:20:48.480669 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:20:48.480807 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:20:48.480875 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0809 16:20:48.481041 (+ 166us) server_negotiation.cc:299] Negotiation successful
0809 16:20:48.481114 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260809 16:20:48.481935 31258 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.21.51.189:41439: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "e438fd6b8e6645f7addce3b7676caca0"
member_type: NON_VOTER
last_known_addr {
host: "127.21.51.189"
port: 41439
}
attrs {
promote: true
}
}
dest_uuid: "4896338a36d9400f84bedb07f140d7ad"
cas_config_opid_index: -1
I20260809 16:20:48.482316 31258 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4896338a36d9400f84bedb07f140d7ad [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "4896338a36d9400f84bedb07f140d7ad" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 40451 } } peers { permanent_uuid: "e438fd6b8e6645f7addce3b7676caca0" member_type: NON_VOTER last_known_addr { host: "127.21.51.189" port: 41439 } attrs { promote: true } }
I20260809 16:20:48.482828 32124 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 4896338a36d9400f84bedb07f140d7ad [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER e438fd6b8e6645f7addce3b7676caca0 (127.21.51.189) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "4896338a36d9400f84bedb07f140d7ad" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 40451 } } peers { permanent_uuid: "e438fd6b8e6645f7addce3b7676caca0" member_type: NON_VOTER last_known_addr { host: "127.21.51.189" port: 41439 } attrs { promote: true } } }
I20260809 16:20:48.483147 32039 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e438fd6b8e6645f7addce3b7676caca0 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:20:48.483979 32124 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.21.51.189:41439
I20260809 16:20:48.484104 32039 raft_consensus.cc:1261] T 00000000000000000000000000000000 P e438fd6b8e6645f7addce3b7676caca0 [term 1 FOLLOWER]: Refusing update from remote peer 4896338a36d9400f84bedb07f140d7ad: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260809 16:20:48.484148 32124 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4896338a36d9400f84bedb07f140d7ad [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "4896338a36d9400f84bedb07f140d7ad" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "4896338a36d9400f84bedb07f140d7ad" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 40451 } } peers { permanent_uuid: "e438fd6b8e6645f7addce3b7676caca0" member_type: NON_VOTER last_known_addr { host: "127.21.51.189" port: 41439 } attrs { promote: true } } }
I20260809 16:20:48.484207 32124 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4896338a36d9400f84bedb07f140d7ad [sys.catalog]: This master's current role is: LEADER
I20260809 16:20:48.484270 31980 master.cc:537] Master@127.21.51.189:41439 shutting down...
I20260809 16:20:48.484408 32125 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 4896338a36d9400f84bedb07f140d7ad [LEADER]: Connected to new peer: Peer: permanent_uuid: "e438fd6b8e6645f7addce3b7676caca0" member_type: NON_VOTER last_known_addr { host: "127.21.51.189" port: 41439 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
W20260809 16:20:48.485172 31228 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
W20260809 16:20:48.485611 31228 consensus_peers.cc:487] T 00000000000000000000000000000000 P 4896338a36d9400f84bedb07f140d7ad -> Peer e438fd6b8e6645f7addce3b7676caca0 (127.21.51.189:41439): Couldn't send request to peer e438fd6b8e6645f7addce3b7676caca0. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260809 16:20:48.487288 31980 raft_consensus.cc:2229] T 00000000000000000000000000000000 P e438fd6b8e6645f7addce3b7676caca0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 16:20:48.487382 31980 raft_consensus.cc:2258] T 00000000000000000000000000000000 P e438fd6b8e6645f7addce3b7676caca0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 16:20:48.487447 31980 tablet_replica.cc:330] T 00000000000000000000000000000000 P e438fd6b8e6645f7addce3b7676caca0: stopping tablet replica
I20260809 16:20:48.500433 31980 master.cc:552] Master@127.21.51.189:41439 shutdown complete.
I20260809 16:20:48.500485 31980 master_runner.cc:305] Clearing existing system tablet
I20260809 16:20:48.500618 31980 fs_manager.cc:359] Metadata directory not provided
I20260809 16:20:48.500672 31980 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:20:48.501436 31980 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.000s sys 0.001s
I20260809 16:20:48.501963 32133 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:20:48.502097 31980 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292417722232-21710-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:20:48.502162 31980 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 16:20:48.502199 31980 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292417722232-21710-0/minicluster-data/master-1/data,/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292417722232-21710-0/minicluster-data/master-1/wal
uuid: "e438fd6b8e6645f7addce3b7676caca0"
format_stamp: "Formatted at 2026-08-09 16:20:48 on dist-test-slave-s11t"
server_key: "ad1fb6f1e5c20f8256f04d770743d2a4"
server_key_iv: "1c992898d34a3059b986f184db127a42"
server_key_version: "encryptionkey@0"
I20260809 16:20:48.502856 31980 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P e438fd6b8e6645f7addce3b7676caca0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 16:20:48.504200 31980 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P e438fd6b8e6645f7addce3b7676caca0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260809 16:20:48.504261 31980 log.cc:1196] T 00000000000000000000000000000000 P e438fd6b8e6645f7addce3b7676caca0: Deleting WAL directory at /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292417722232-21710-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260809 16:20:48.504458 31980 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P e438fd6b8e6645f7addce3b7676caca0: Deleting consensus metadata
I20260809 16:20:48.504591 31980 master_runner.cc:315] Copying system tablet from 127.21.51.190:40451
I20260809 16:20:48.505256 31980 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P e438fd6b8e6645f7addce3b7676caca0: tablet copy: Beginning tablet copy session from remote peer at address 127.21.51.190:40451
I20260809 16:20:48.511287 31325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:48.505604 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40110 (local address 127.21.51.190:40451)
0809 16:20:48.505657 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0809 16:20:48.505659 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:20:48.505717 (+ 58us) server_negotiation.cc:407] Connection header received
0809 16:20:48.505795 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:20:48.505797 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:20:48.505807 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:20:48.505864 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:20:48.506206 (+ 342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:48.506705 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:48.507443 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:48.507642 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:48.507732 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:48.508761 (+ 1029us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:20:48.508763 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:20:48.508763 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:20:48.508771 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:20:48.510096 (+ 1325us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:20:48.510476 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:20:48.510477 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:20:48.510478 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:20:48.510542 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:20:48.510749 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:20:48.510750 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:20:48.510750 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:20:48.510867 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:20:48.510947 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0809 16:20:48.511127 (+ 180us) server_negotiation.cc:299] Negotiation successful
0809 16:20:48.511200 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260809 16:20:48.511308 32144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:48.505527 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:40451 (local address 127.0.0.1:40110)
0809 16:20:48.505657 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0809 16:20:48.505669 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 16:20:48.505734 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:20:48.505871 (+ 137us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:20:48.505871 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:20:48.506067 (+ 196us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:20:48.506121 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:20:48.506123 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:20:48.506797 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:20:48.506797 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:20:48.507350 (+ 553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:20:48.507352 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:20:48.507733 (+ 381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:20:48.507734 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:20:48.507806 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:48.508294 (+ 488us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:20:48.508299 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:20:48.508599 (+ 300us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:20:48.510181 (+ 1582us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:20:48.510182 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:20:48.510183 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:20:48.510365 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:20:48.510618 (+ 253us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:20:48.510619 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:20:48.510619 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:20:48.510684 (+ 65us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:20:48.510939 (+ 255us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:20:48.510940 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:20:48.510984 (+ 44us) client_negotiation.cc:769] Sending connection context
0809 16:20:48.511083 (+ 99us) client_negotiation.cc:240] Negotiation successful
0809 16:20:48.511196 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":87,"threads_started":1}
I20260809 16:20:48.511869 31290 tablet_copy_service.cc:140] P 4896338a36d9400f84bedb07f140d7ad: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer e438fd6b8e6645f7addce3b7676caca0 ({username='oryx', principal='oryx/127.21.51.189@KRBTEST.COM'} at 127.0.0.1:40110)
I20260809 16:20:48.511983 31290 tablet_copy_service.cc:161] P 4896338a36d9400f84bedb07f140d7ad: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer e438fd6b8e6645f7addce3b7676caca0 at {username='oryx', principal='oryx/127.21.51.189@KRBTEST.COM'} at 127.0.0.1:40110: session id = e438fd6b8e6645f7addce3b7676caca0-00000000000000000000000000000000
I20260809 16:20:48.512677 31290 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 4896338a36d9400f84bedb07f140d7ad: Tablet Copy: opened 0 blocks and 1 log segments
I20260809 16:20:48.513362 31980 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260809 16:20:48.514303 31980 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P e438fd6b8e6645f7addce3b7676caca0: tablet copy: Starting download of 0 data blocks...
I20260809 16:20:48.514452 31980 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P e438fd6b8e6645f7addce3b7676caca0: tablet copy: Starting download of 1 WAL segments...
I20260809 16:20:48.515373 31980 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P e438fd6b8e6645f7addce3b7676caca0: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260809 16:20:48.516296 31290 tablet_copy_service.cc:342] P 4896338a36d9400f84bedb07f140d7ad: Request end of tablet copy session e438fd6b8e6645f7addce3b7676caca0-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.21.51.189@KRBTEST.COM'} at 127.0.0.1:40110
I20260809 16:20:48.516356 31290 tablet_copy_service.cc:433] P 4896338a36d9400f84bedb07f140d7ad: ending tablet copy session e438fd6b8e6645f7addce3b7676caca0-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer e438fd6b8e6645f7addce3b7676caca0
I20260809 16:20:48.518849 31980 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:20:48.519955 32146 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:20:48.520028 32147 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:20:48.521548 31980 server_base.cc:1034] running on GCE node
W20260809 16:20:48.521611 32151 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:20:48.521746 31980 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:20:48.521942 31980 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:20:48.523087 31980 hybrid_clock.cc:648] HybridClock initialized: now 1786292448523078 us; error 27 us; skew 500 ppm
Aug 09 16:20:48 dist-test-slave-s11t krb5kdc[31184](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292448, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.51.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:20:48.524472 31980 init.cc:377] Logged in from keytab as oryx/127.21.51.189@KRBTEST.COM (short username oryx)
I20260809 16:20:48.524739 31980 webserver.cc:460] Webserver started at http://127.21.51.189:35763/ using document root <none> and password file <none>
I20260809 16:20:48.524864 31980 fs_manager.cc:359] Metadata directory not provided
I20260809 16:20:48.524902 31980 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:20:48.525588 31980 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.000s sys 0.001s
I20260809 16:20:48.526151 32164 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:20:48.526269 31980 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 16:20:48.526319 31980 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292417722232-21710-0/minicluster-data/master-1/data,/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292417722232-21710-0/minicluster-data/master-1/wal
uuid: "e438fd6b8e6645f7addce3b7676caca0"
format_stamp: "Formatted at 2026-08-09 16:20:48 on dist-test-slave-s11t"
server_key: "ad1fb6f1e5c20f8256f04d770743d2a4"
server_key_iv: "1c992898d34a3059b986f184db127a42"
server_key_version: "encryptionkey@0"
I20260809 16:20:48.526371 31980 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292417722232-21710-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292417722232-21710-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292417722232-21710-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:20:48.537209 31980 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:20:48.538728 31980 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:20:48.541893 31980 rpc_server.cc:307] RPC server started. Bound to: 127.21.51.189:41439
I20260809 16:20:48.542109 32236 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.51.189:41439 every 8 connection(s)
I20260809 16:20:48.542303 31980 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292417722232-21710-0/minicluster-data/master-1/data/info.pb
I20260809 16:20:48.543059 32237 sys_catalog.cc:263] Verifying existing consensus state
I20260809 16:20:48.543496 32237 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e438fd6b8e6645f7addce3b7676caca0: Bootstrap starting.
I20260809 16:20:48.545804 32237 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e438fd6b8e6645f7addce3b7676caca0: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 16:20:48.546051 32237 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e438fd6b8e6645f7addce3b7676caca0: Bootstrap complete.
I20260809 16:20:48.546253 32237 raft_consensus.cc:348] T 00000000000000000000000000000000 P e438fd6b8e6645f7addce3b7676caca0 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "4896338a36d9400f84bedb07f140d7ad" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 40451 } } peers { permanent_uuid: "e438fd6b8e6645f7addce3b7676caca0" member_type: NON_VOTER last_known_addr { host: "127.21.51.189" port: 41439 } attrs { promote: true } }
I20260809 16:20:48.546320 32237 raft_consensus.cc:729] T 00000000000000000000000000000000 P e438fd6b8e6645f7addce3b7676caca0 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: e438fd6b8e6645f7addce3b7676caca0, State: Initialized, Role: LEARNER
I20260809 16:20:48.546346 32237 consensus_queue.cc:260] T 00000000000000000000000000000000 P e438fd6b8e6645f7addce3b7676caca0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "4896338a36d9400f84bedb07f140d7ad" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 40451 } } peers { permanent_uuid: "e438fd6b8e6645f7addce3b7676caca0" member_type: NON_VOTER last_known_addr { host: "127.21.51.189" port: 41439 } attrs { promote: true } }
I20260809 16:20:48.546646 32237 sys_catalog.cc:564] T 00000000000000000000000000000000 P e438fd6b8e6645f7addce3b7676caca0 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 16:20:48.546646 32243 sys_catalog.cc:455] T 00000000000000000000000000000000 P e438fd6b8e6645f7addce3b7676caca0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "4896338a36d9400f84bedb07f140d7ad" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 40451 } } peers { permanent_uuid: "e438fd6b8e6645f7addce3b7676caca0" member_type: NON_VOTER last_known_addr { host: "127.21.51.189" port: 41439 } attrs { promote: true } } }
I20260809 16:20:48.546777 32243 sys_catalog.cc:458] T 00000000000000000000000000000000 P e438fd6b8e6645f7addce3b7676caca0 [sys.catalog]: This master's current role is: LEARNER
I20260809 16:20:48.548074 32261 catalog_manager.cc:1259] Loaded cluster ID: 70b9d0b7af95488b90d1785aca0f4ff6
I20260809 16:20:48.548122 32261 catalog_manager.cc:1552] T 00000000000000000000000000000000 P e438fd6b8e6645f7addce3b7676caca0: loading cluster ID for follower catalog manager: success
I20260809 16:20:48.549383 32261 catalog_manager.cc:1574] T 00000000000000000000000000000000 P e438fd6b8e6645f7addce3b7676caca0: acquiring CA information for follower catalog manager: success
I20260809 16:20:48.549655 32261 catalog_manager.cc:1602] T 00000000000000000000000000000000 P e438fd6b8e6645f7addce3b7676caca0: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 16:20:48.556521 32264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:48.549186 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48392 (local address 127.21.51.189:41439)
0809 16:20:48.549789 (+ 603us) server_negotiation.cc:206] Beginning negotiation
0809 16:20:48.549791 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:20:48.549800 (+ 9us) server_negotiation.cc:407] Connection header received
0809 16:20:48.549824 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:20:48.549825 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:20:48.549835 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:20:48.549919 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:20:48.550591 (+ 672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:48.551304 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:48.552350 (+ 1046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:48.552546 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:48.552600 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:48.553979 (+ 1379us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:20:48.553980 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:20:48.553980 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:20:48.553986 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:20:48.555401 (+ 1415us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:20:48.555813 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:20:48.555814 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:20:48.555814 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:20:48.555868 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:20:48.556081 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:20:48.556082 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:20:48.556082 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:20:48.556176 (+ 94us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:20:48.556248 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0809 16:20:48.556399 (+ 151us) server_negotiation.cc:299] Negotiation successful
0809 16:20:48.556459 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":558,"thread_start_us":46,"threads_started":1}
WARNING: no policy specified for oryx/127.21.51.188@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.21.51.188@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.21.51.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292417722232-21710-0/krb5kdc/oryx_127.21.51.188.keytab.
Entry for principal oryx/127.21.51.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292417722232-21710-0/krb5kdc/oryx_127.21.51.188.keytab.
I20260809 16:20:48.581478 21710 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrNqUCn/build/release/bin/kudu
/tmp/dist-test-taskrNqUCn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292417722232-21710-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292417722232-21710-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292417722232-21710-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292417722232-21710-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.51.188:40073
--webserver_interface=127.21.51.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292417722232-21710-0/krb5kdc/oryx_127.21.51.188.keytab
--principal=oryx/127.21.51.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.51.148:33645
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.51.190:40451,127.21.51.189:41439,127.21.51.188:40073
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292417722232-21710-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292417722232-21710-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292417722232-21710-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:20:48.664566 32270 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:20:48.664819 32270 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:20:48.664845 32270 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:20:48.665983 32270 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 16:20:48.666024 32270 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:20:48.666038 32270 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:20:48.666051 32270 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 16:20:48.666109 32270 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 16:20:48.667572 32270 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.51.148:33645
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292417722232-21710-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292417722232-21710-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.21.51.190:40451,127.21.51.189:41439,127.21.51.188:40073
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292417722232-21710-0/krb5kdc/oryx_127.21.51.188.keytab
--principal=oryx/127.21.51.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.51.188:40073
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292417722232-21710-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.21.51.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32270
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292417722232-21710-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:20:48.667819 32270 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:20:48.668071 32270 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:20:48.672901 32285 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:20:48.672914 32282 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:20:48.672912 32281 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:20:48.673210 32270 server_base.cc:1034] running on GCE node
I20260809 16:20:48.673424 32270 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:20:48.673648 32270 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:20:48.674836 32270 hybrid_clock.cc:648] HybridClock initialized: now 1786292448674824 us; error 28 us; skew 500 ppm
Aug 09 16:20:48 dist-test-slave-s11t krb5kdc[31184](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292448, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.51.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:20:48.679876 32270 init.cc:377] Logged in from keytab as oryx/127.21.51.188@KRBTEST.COM (short username oryx)
I20260809 16:20:48.680259 32270 webserver.cc:460] Webserver started at http://127.21.51.188:45545/ using document root <none> and password file <none>
I20260809 16:20:48.680436 32270 fs_manager.cc:359] Metadata directory not provided
I20260809 16:20:48.680476 32270 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:20:48.680563 32270 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:20:48.681622 32270 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292417722232-21710-0/minicluster-data/master-2/data/instance:
uuid: "04e0582b6c844b10a2f992e33317c169"
format_stamp: "Formatted at 2026-08-09 16:20:48 on dist-test-slave-s11t"
server_key: "05ea5f561327bbb8aafb5007706daf6b"
server_key_iv: "ffa73ae1ea8fa325283b8105c82ecefc"
server_key_version: "encryptionkey@0"
I20260809 16:20:48.681932 32270 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292417722232-21710-0/minicluster-data/master-2/wal/instance:
uuid: "04e0582b6c844b10a2f992e33317c169"
format_stamp: "Formatted at 2026-08-09 16:20:48 on dist-test-slave-s11t"
server_key: "05ea5f561327bbb8aafb5007706daf6b"
server_key_iv: "ffa73ae1ea8fa325283b8105c82ecefc"
server_key_version: "encryptionkey@0"
I20260809 16:20:48.683192 32270 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 16:20:48.683914 32294 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:20:48.684094 32270 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:20:48.684196 32270 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292417722232-21710-0/minicluster-data/master-2/data,/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292417722232-21710-0/minicluster-data/master-2/wal
uuid: "04e0582b6c844b10a2f992e33317c169"
format_stamp: "Formatted at 2026-08-09 16:20:48 on dist-test-slave-s11t"
server_key: "05ea5f561327bbb8aafb5007706daf6b"
server_key_iv: "ffa73ae1ea8fa325283b8105c82ecefc"
server_key_version: "encryptionkey@0"
I20260809 16:20:48.684273 32270 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292417722232-21710-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292417722232-21710-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292417722232-21710-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:20:48.694764 32270 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:20:48.696768 32270 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:20:48.696923 32270 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:20:48.700971 32270 rpc_server.cc:307] RPC server started. Bound to: 127.21.51.188:40073
I20260809 16:20:48.701005 32372 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.51.188:40073 every 8 connection(s)
I20260809 16:20:48.701339 32270 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292417722232-21710-0/minicluster-data/master-2/data/info.pb
I20260809 16:20:48.702296 32373 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:20:48.703820 32373 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.51.190" port: 40451 } has no permanent_uuid. Determining permanent_uuid...
I20260809 16:20:48.706583 21710 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrNqUCn/build/release/bin/kudu as pid 32270
Aug 09 16:20:48 dist-test-slave-s11t krb5kdc[31184](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292448, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.51.188@KRBTEST.COM for oryx/127.21.51.190@KRBTEST.COM
Aug 09 16:20:48 dist-test-slave-s11t krb5kdc[31184](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292447, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.21.51.188@KRBTEST.COM
I20260809 16:20:48.713673 31325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:48.704653 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40116 (local address 127.21.51.190:40451)
0809 16:20:48.704747 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0809 16:20:48.704749 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:20:48.704814 (+ 65us) server_negotiation.cc:407] Connection header received
0809 16:20:48.705313 (+ 499us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:20:48.705314 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:20:48.705327 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:20:48.705375 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:20:48.706004 (+ 629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:48.706505 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:48.707898 (+ 1393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:48.708102 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:48.708186 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:48.710851 (+ 2665us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:20:48.710853 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:20:48.710854 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:20:48.710861 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:20:48.712169 (+ 1308us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:20:48.712632 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:20:48.712632 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:20:48.712633 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:20:48.712730 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:20:48.713016 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:20:48.713017 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:20:48.713017 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:20:48.713135 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:20:48.713206 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0809 16:20:48.713487 (+ 281us) server_negotiation.cc:299] Negotiation successful
0809 16:20:48.713585 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260809 16:20:48.713909 32376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:48.704466 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:40451 (local address 127.0.0.1:40116)
0809 16:20:48.704653 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0809 16:20:48.704741 (+ 88us) client_negotiation.cc:174] Beginning negotiation
0809 16:20:48.705202 (+ 461us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:20:48.705473 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:20:48.705476 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:20:48.705714 (+ 238us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:20:48.705905 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:20:48.705909 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:20:48.706607 (+ 698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:20:48.706608 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:20:48.707811 (+ 1203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:20:48.707812 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:20:48.708232 (+ 420us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:20:48.708233 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:20:48.708356 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:48.708899 (+ 543us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:20:48.708907 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:20:48.710734 (+ 1827us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:20:48.712269 (+ 1535us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:20:48.712276 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:20:48.712278 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:20:48.712500 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:20:48.712820 (+ 320us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:20:48.712821 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:20:48.712821 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:20:48.712939 (+ 118us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:20:48.713208 (+ 269us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:20:48.713215 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:20:48.713350 (+ 135us) client_negotiation.cc:769] Sending connection context
0809 16:20:48.713448 (+ 98us) client_negotiation.cc:240] Negotiation successful
0809 16:20:48.713630 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":84,"threads_started":1}
I20260809 16:20:48.714558 32373 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.51.189" port: 41439 } has no permanent_uuid. Determining permanent_uuid...
I20260809 16:20:48.716173 32377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:48.707213 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44154 (local address 127.21.51.188:40073)
0809 16:20:48.707444 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0809 16:20:48.707449 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 16:20:48.707470 (+ 21us) server_negotiation.cc:407] Connection header received
0809 16:20:48.707492 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:20:48.707494 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:20:48.707513 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:20:48.707596 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:20:48.708170 (+ 574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:48.709061 (+ 891us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:48.709924 (+ 863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:48.710389 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:48.710445 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:48.713051 (+ 2606us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:20:48.713058 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:20:48.713059 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:20:48.713068 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:20:48.714901 (+ 1833us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:20:48.715346 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:20:48.715350 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:20:48.715350 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:20:48.715442 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:20:48.715645 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:20:48.715646 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:20:48.715646 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:20:48.715808 (+ 162us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:20:48.715944 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0809 16:20:48.715985 (+ 41us) server_negotiation.cc:299] Negotiation successful
0809 16:20:48.716058 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":201,"thread_start_us":153,"threads_started":1}
Aug 09 16:20:48 dist-test-slave-s11t krb5kdc[31184](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292448, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.51.188@KRBTEST.COM for oryx/127.21.51.189@KRBTEST.COM
I20260809 16:20:48.721271 32376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:48.714805 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.189:41439 (local address 127.0.0.1:48406)
0809 16:20:48.714859 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0809 16:20:48.714870 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 16:20:48.714912 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:20:48.715086 (+ 174us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:20:48.715087 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:20:48.715228 (+ 141us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:20:48.715303 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:20:48.715305 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:20:48.716202 (+ 897us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:20:48.716203 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:20:48.716974 (+ 771us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:20:48.716977 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:20:48.717328 (+ 351us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:20:48.717329 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:20:48.717405 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:48.717773 (+ 368us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:20:48.717777 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:20:48.718864 (+ 1087us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:20:48.720464 (+ 1600us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:20:48.720465 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:20:48.720465 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:20:48.720610 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:20:48.720835 (+ 225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:20:48.720836 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:20:48.720836 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:20:48.720889 (+ 53us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:20:48.721071 (+ 182us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:20:48.721072 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:20:48.721104 (+ 32us) client_negotiation.cc:769] Sending connection context
0809 16:20:48.721153 (+ 49us) client_negotiation.cc:240] Negotiation successful
0809 16:20:48.721218 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35}
I20260809 16:20:48.721293 32264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:48.714890 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48406 (local address 127.21.51.189:41439)
0809 16:20:48.714963 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0809 16:20:48.714965 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:20:48.714975 (+ 10us) server_negotiation.cc:407] Connection header received
0809 16:20:48.715002 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:20:48.715003 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:20:48.715016 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:20:48.715075 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:20:48.715362 (+ 287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:48.716108 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:48.717051 (+ 943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:48.717261 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:48.717317 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:48.718947 (+ 1630us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:20:48.718949 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:20:48.718949 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:20:48.718955 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:20:48.720397 (+ 1442us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:20:48.720663 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:20:48.720663 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:20:48.720663 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:20:48.720759 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:20:48.720931 (+ 172us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:20:48.720931 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:20:48.720932 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:20:48.721023 (+ 91us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:20:48.721068 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0809 16:20:48.721168 (+ 100us) server_negotiation.cc:299] Negotiation successful
0809 16:20:48.721233 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260809 16:20:48.721799 32373 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.51.188" port: 40073 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 16:20:48 dist-test-slave-s11t krb5kdc[31184](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292448, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.51.188@KRBTEST.COM for oryx/127.21.51.188@KRBTEST.COM
I20260809 16:20:48.727730 32376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:48.722034 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.188:40073 (local address 127.0.0.1:44162)
0809 16:20:48.722124 (+ 90us) negotiation.cc:107] Waiting for socket to connect
0809 16:20:48.722140 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 16:20:48.722179 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:20:48.722258 (+ 79us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:20:48.722258 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:20:48.722378 (+ 120us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:20:48.722403 (+ 25us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:20:48.722404 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:20:48.722967 (+ 563us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:20:48.722968 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:20:48.723418 (+ 450us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:20:48.723419 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:20:48.723719 (+ 300us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:20:48.723719 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:20:48.723754 (+ 35us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:48.724119 (+ 365us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:20:48.724123 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:20:48.725286 (+ 1163us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:20:48.726747 (+ 1461us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:20:48.726748 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:20:48.726748 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:20:48.726966 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:20:48.727134 (+ 168us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:20:48.727134 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:20:48.727135 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:20:48.727185 (+ 50us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:20:48.727428 (+ 243us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:20:48.727428 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:20:48.727465 (+ 37us) client_negotiation.cc:769] Sending connection context
0809 16:20:48.727533 (+ 68us) client_negotiation.cc:240] Negotiation successful
0809 16:20:48.727622 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":77}
I20260809 16:20:48.727784 32377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:48.722087 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44162 (local address 127.21.51.188:40073)
0809 16:20:48.722116 (+ 29us) server_negotiation.cc:206] Beginning negotiation
0809 16:20:48.722118 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:20:48.722168 (+ 50us) server_negotiation.cc:407] Connection header received
0809 16:20:48.722213 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:20:48.722213 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:20:48.722220 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:20:48.722256 (+ 36us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:20:48.722458 (+ 202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:48.722899 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:48.723482 (+ 583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:48.723660 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:48.723707 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:48.725359 (+ 1652us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:20:48.725360 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:20:48.725360 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:20:48.725366 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:20:48.726681 (+ 1315us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:20:48.727030 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:20:48.727031 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:20:48.727031 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:20:48.727078 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:20:48.727252 (+ 174us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:20:48.727252 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:20:48.727252 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:20:48.727353 (+ 101us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:20:48.727435 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0809 16:20:48.727560 (+ 125us) server_negotiation.cc:299] Negotiation successful
0809 16:20:48.727670 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":52,"server-negotiator.queue_time_us":15}
I20260809 16:20:48.729147 32373 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 04e0582b6c844b10a2f992e33317c169: Bootstrap starting.
I20260809 16:20:48.729945 32373 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 04e0582b6c844b10a2f992e33317c169: Neither blocks nor log segments found. Creating new log.
I20260809 16:20:48.730242 32373 log.cc:824] T 00000000000000000000000000000000 P 04e0582b6c844b10a2f992e33317c169: Log is configured to *not* fsync() on all Append() calls
I20260809 16:20:48.730960 32373 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 04e0582b6c844b10a2f992e33317c169: No bootstrap required, opened a new log
I20260809 16:20:48.732339 32373 raft_consensus.cc:348] T 00000000000000000000000000000000 P 04e0582b6c844b10a2f992e33317c169 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4896338a36d9400f84bedb07f140d7ad" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 40451 } } peers { permanent_uuid: "e438fd6b8e6645f7addce3b7676caca0" member_type: VOTER last_known_addr { host: "127.21.51.189" port: 41439 } } peers { permanent_uuid: "04e0582b6c844b10a2f992e33317c169" member_type: VOTER last_known_addr { host: "127.21.51.188" port: 40073 } }
I20260809 16:20:48.732458 32373 raft_consensus.cc:374] T 00000000000000000000000000000000 P 04e0582b6c844b10a2f992e33317c169 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:20:48.732481 32373 raft_consensus.cc:729] T 00000000000000000000000000000000 P 04e0582b6c844b10a2f992e33317c169 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 04e0582b6c844b10a2f992e33317c169, State: Initialized, Role: FOLLOWER
I20260809 16:20:48.732630 32373 consensus_queue.cc:260] T 00000000000000000000000000000000 P 04e0582b6c844b10a2f992e33317c169 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4896338a36d9400f84bedb07f140d7ad" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 40451 } } peers { permanent_uuid: "e438fd6b8e6645f7addce3b7676caca0" member_type: VOTER last_known_addr { host: "127.21.51.189" port: 41439 } } peers { permanent_uuid: "04e0582b6c844b10a2f992e33317c169" member_type: VOTER last_known_addr { host: "127.21.51.188" port: 40073 } }
I20260809 16:20:48.733032 32373 sys_catalog.cc:564] T 00000000000000000000000000000000 P 04e0582b6c844b10a2f992e33317c169 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 16:20:48.732980 32380 sys_catalog.cc:455] T 00000000000000000000000000000000 P 04e0582b6c844b10a2f992e33317c169 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4896338a36d9400f84bedb07f140d7ad" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 40451 } } peers { permanent_uuid: "e438fd6b8e6645f7addce3b7676caca0" member_type: VOTER last_known_addr { host: "127.21.51.189" port: 41439 } } peers { permanent_uuid: "04e0582b6c844b10a2f992e33317c169" member_type: VOTER last_known_addr { host: "127.21.51.188" port: 40073 } } }
I20260809 16:20:48.733093 32380 sys_catalog.cc:458] T 00000000000000000000000000000000 P 04e0582b6c844b10a2f992e33317c169 [sys.catalog]: This master's current role is: FOLLOWER
W20260809 16:20:48.734714 32398 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 04e0582b6c844b10a2f992e33317c169: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 16:20:48.734776 32398 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 16:20:48.737005 32270 master_runner.cc:428] Detected that this master 04e0582b6c844b10a2f992e33317c169 is joining an existing cluster
I20260809 16:20:48.737062 32270 master_runner.cc:432] Initiating AddMaster RPC to add 127.21.51.188:40073
I20260809 16:20:48.761705 32264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:48.738430 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48408 (local address 127.21.51.189:41439)
0809 16:20:48.738469 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0809 16:20:48.738471 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:20:48.738571 (+ 100us) server_negotiation.cc:407] Connection header received
0809 16:20:48.738666 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:20:48.738667 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:20:48.738677 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:20:48.738708 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:20:48.739106 (+ 398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:48.739806 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:48.740786 (+ 980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:48.741070 (+ 284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:48.741111 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:48.742221 (+ 1110us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:20:48.742222 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:20:48.742222 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:20:48.742230 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:20:48.743743 (+ 1513us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:20:48.744130 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:20:48.744131 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:20:48.744131 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:20:48.744192 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:20:48.761067 (+ 16875us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:20:48.761069 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:20:48.761069 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:20:48.761227 (+ 158us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:20:48.761275 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0809 16:20:48.761505 (+ 230us) server_negotiation.cc:299] Negotiation successful
0809 16:20:48.761606 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260809 16:20:48.762277 32413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:48.738247 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.189:41439 (local address 127.0.0.1:48408)
0809 16:20:48.738516 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0809 16:20:48.738525 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 16:20:48.738594 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:20:48.738773 (+ 179us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:20:48.738774 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:20:48.738970 (+ 196us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:20:48.739019 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:20:48.739021 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:20:48.739892 (+ 871us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:20:48.739893 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:20:48.740655 (+ 762us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:20:48.740657 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:20:48.741207 (+ 550us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:20:48.741208 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:20:48.741280 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:48.741761 (+ 481us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:20:48.741765 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:20:48.742107 (+ 342us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:20:48.743828 (+ 1721us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:20:48.743829 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:20:48.743830 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:20:48.744047 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:20:48.744267 (+ 220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:20:48.744267 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:20:48.744268 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:20:48.760953 (+ 16685us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:20:48.761325 (+ 372us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:20:48.761326 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:20:48.761391 (+ 65us) client_negotiation.cc:769] Sending connection context
0809 16:20:48.761807 (+ 416us) client_negotiation.cc:240] Negotiation successful
0809 16:20:48.762158 (+ 351us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":255,"thread_start_us":60,"threads_started":1}
I20260809 16:20:48.764544 32410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:48.738002 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.188:40073 (local address 127.0.0.1:44164)
0809 16:20:48.738345 (+ 343us) negotiation.cc:107] Waiting for socket to connect
0809 16:20:48.738353 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0809 16:20:48.738403 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:20:48.738551 (+ 148us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:20:48.738551 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:20:48.738776 (+ 225us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:20:48.738835 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:20:48.738837 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:20:48.739698 (+ 861us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:20:48.739699 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:20:48.743493 (+ 3794us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:20:48.743497 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:20:48.743981 (+ 484us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:20:48.743982 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:20:48.744052 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:48.744892 (+ 840us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:20:48.744896 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:20:48.761369 (+ 16473us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:20:48.763378 (+ 2009us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:20:48.763379 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:20:48.763380 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:20:48.763601 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:20:48.763855 (+ 254us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:20:48.763856 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:20:48.763856 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:20:48.763917 (+ 61us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:20:48.764178 (+ 261us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:20:48.764179 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:20:48.764217 (+ 38us) client_negotiation.cc:769] Sending connection context
0809 16:20:48.764304 (+ 87us) client_negotiation.cc:240] Negotiation successful
0809 16:20:48.764415 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":329,"spinlock_wait_cycles":8448,"thread_start_us":109,"threads_started":1}
I20260809 16:20:48.764603 32377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:48.738209 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44164 (local address 127.21.51.188:40073)
0809 16:20:48.738382 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0809 16:20:48.738384 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:20:48.738392 (+ 8us) server_negotiation.cc:407] Connection header received
0809 16:20:48.738469 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:20:48.738470 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:20:48.738477 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:20:48.738502 (+ 25us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:20:48.738928 (+ 426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:48.739611 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:48.743590 (+ 3979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:48.743896 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:48.743964 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:48.761497 (+ 17533us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:20:48.761498 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:20:48.761499 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:20:48.761505 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:20:48.763291 (+ 1786us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:20:48.763703 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:20:48.763704 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:20:48.763705 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:20:48.763769 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:20:48.763992 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:20:48.763993 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:20:48.763993 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:20:48.764103 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:20:48.764174 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0809 16:20:48.764337 (+ 163us) server_negotiation.cc:299] Negotiation successful
0809 16:20:48.764467 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":66,"server-negotiator.queue_time_us":155}
I20260809 16:20:48.765630 31325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:48.738028 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40122 (local address 127.21.51.190:40451)
0809 16:20:48.738071 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0809 16:20:48.738074 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 16:20:48.738231 (+ 157us) server_negotiation.cc:407] Connection header received
0809 16:20:48.738318 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:20:48.738319 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:20:48.738330 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:20:48.738398 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:20:48.738876 (+ 478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:48.739640 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:48.740811 (+ 1171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:48.741114 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:48.741153 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:48.762632 (+ 21479us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:20:48.762634 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:20:48.762634 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:20:48.762643 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:20:48.764394 (+ 1751us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:20:48.764828 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:20:48.764830 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:20:48.764830 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:20:48.764897 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:20:48.765093 (+ 196us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:20:48.765094 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:20:48.765095 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:20:48.765220 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:20:48.765271 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0809 16:20:48.765480 (+ 209us) server_negotiation.cc:299] Negotiation successful
0809 16:20:48.765563 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260809 16:20:48.765635 32412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:48.737998 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:40451 (local address 127.0.0.1:40122)
0809 16:20:48.738163 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0809 16:20:48.738173 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 16:20:48.738255 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:20:48.738446 (+ 191us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:20:48.738447 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:20:48.738687 (+ 240us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:20:48.738781 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:20:48.738783 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:20:48.739737 (+ 954us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:20:48.739738 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:20:48.740655 (+ 917us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:20:48.740657 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:20:48.741221 (+ 564us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:20:48.741221 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:20:48.741280 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:48.761447 (+ 20167us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:20:48.761453 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:20:48.762275 (+ 822us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:20:48.764490 (+ 2215us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:20:48.764491 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:20:48.764491 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:20:48.764741 (+ 250us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:20:48.764970 (+ 229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:20:48.764971 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:20:48.764971 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:20:48.765028 (+ 57us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:20:48.765303 (+ 275us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:20:48.765304 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:20:48.765343 (+ 39us) client_negotiation.cc:769] Sending connection context
0809 16:20:48.765425 (+ 82us) client_negotiation.cc:240] Negotiation successful
0809 16:20:48.765527 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":108,"threads_started":1}
I20260809 16:20:48.767498 21710 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrNqUCn/build/release/bin/kudu with pid 31354
Aug 09 16:20:48 dist-test-slave-s11t krb5kdc[31184](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292447, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.51.190@KRBTEST.COM for oryx/127.21.51.188@KRBTEST.COM
I20260809 16:20:48.779419 32123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:48.767839 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.188:40073 (local address 127.0.0.1:44170)
0809 16:20:48.767894 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0809 16:20:48.767903 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 16:20:48.767938 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:20:48.770388 (+ 2450us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:20:48.770389 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:20:48.770549 (+ 160us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:20:48.770594 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:20:48.770596 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:20:48.771305 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:20:48.771306 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:20:48.772274 (+ 968us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:20:48.772278 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:20:48.772934 (+ 656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:20:48.772936 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:20:48.773034 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:48.773528 (+ 494us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:20:48.773534 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:20:48.775520 (+ 1986us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:20:48.777926 (+ 2406us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:20:48.777927 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:20:48.777928 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:20:48.778210 (+ 282us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:20:48.778453 (+ 243us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:20:48.778454 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:20:48.778455 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:20:48.778533 (+ 78us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:20:48.778990 (+ 457us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:20:48.778992 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:20:48.779052 (+ 60us) client_negotiation.cc:769] Sending connection context
0809 16:20:48.779135 (+ 83us) client_negotiation.cc:240] Negotiation successful
0809 16:20:48.779264 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":28}
I20260809 16:20:48.780159 32377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:48.770081 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44170 (local address 127.21.51.188:40073)
0809 16:20:48.770235 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0809 16:20:48.770237 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:20:48.770245 (+ 8us) server_negotiation.cc:407] Connection header received
0809 16:20:48.770277 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:20:48.770278 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:20:48.770287 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:20:48.770339 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:20:48.770660 (+ 321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:48.771237 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:48.772383 (+ 1146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:48.772788 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:48.772874 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:48.775629 (+ 2755us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:20:48.775631 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:20:48.775631 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:20:48.775638 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:20:48.777805 (+ 2167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:20:48.778300 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:20:48.778301 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:20:48.778302 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:20:48.778366 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:20:48.778618 (+ 252us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:20:48.778619 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:20:48.778620 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:20:48.778873 (+ 253us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:20:48.778942 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0809 16:20:48.779920 (+ 978us) server_negotiation.cc:299] Negotiation successful
0809 16:20:48.780017 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":136}
I20260809 16:20:48.780905 31258 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.21.51.188:40073: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "04e0582b6c844b10a2f992e33317c169"
member_type: NON_VOTER
last_known_addr {
host: "127.21.51.188"
port: 40073
}
attrs {
promote: true
}
}
dest_uuid: "4896338a36d9400f84bedb07f140d7ad"
cas_config_opid_index: 5
I20260809 16:20:48.781213 31258 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4896338a36d9400f84bedb07f140d7ad [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "4896338a36d9400f84bedb07f140d7ad" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 40451 } } peers { permanent_uuid: "e438fd6b8e6645f7addce3b7676caca0" member_type: NON_VOTER last_known_addr { host: "127.21.51.189" port: 41439 } attrs { promote: true } } peers { permanent_uuid: "04e0582b6c844b10a2f992e33317c169" member_type: NON_VOTER last_known_addr { host: "127.21.51.188" port: 40073 } attrs { promote: true } }
I20260809 16:20:48.782622 32125 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 4896338a36d9400f84bedb07f140d7ad [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 04e0582b6c844b10a2f992e33317c169 (127.21.51.188) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "4896338a36d9400f84bedb07f140d7ad" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 40451 } } peers { permanent_uuid: "e438fd6b8e6645f7addce3b7676caca0" member_type: NON_VOTER last_known_addr { host: "127.21.51.189" port: 41439 } attrs { promote: true } } peers { permanent_uuid: "04e0582b6c844b10a2f992e33317c169" member_type: NON_VOTER last_known_addr { host: "127.21.51.188" port: 40073 } attrs { promote: true } } }
I20260809 16:20:48.784036 32125 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.21.51.188:40073
I20260809 16:20:48.784117 32125 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4896338a36d9400f84bedb07f140d7ad [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "4896338a36d9400f84bedb07f140d7ad" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "4896338a36d9400f84bedb07f140d7ad" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 40451 } } peers { permanent_uuid: "e438fd6b8e6645f7addce3b7676caca0" member_type: NON_VOTER last_known_addr { host: "127.21.51.189" port: 41439 } attrs { promote: true } } peers { permanent_uuid: "04e0582b6c844b10a2f992e33317c169" member_type: NON_VOTER last_known_addr { host: "127.21.51.188" port: 40073 } attrs { promote: true } } }
I20260809 16:20:48.784211 32125 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4896338a36d9400f84bedb07f140d7ad [sys.catalog]: This master's current role is: LEADER
I20260809 16:20:48.788161 32270 master.cc:537] Master@127.21.51.188:40073 shutting down...
I20260809 16:20:48.792944 32380 raft_consensus.cc:482] T 00000000000000000000000000000000 P 04e0582b6c844b10a2f992e33317c169 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 16:20:48.793072 32380 raft_consensus.cc:504] T 00000000000000000000000000000000 P 04e0582b6c844b10a2f992e33317c169 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4896338a36d9400f84bedb07f140d7ad" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 40451 } } peers { permanent_uuid: "e438fd6b8e6645f7addce3b7676caca0" member_type: VOTER last_known_addr { host: "127.21.51.189" port: 41439 } } peers { permanent_uuid: "04e0582b6c844b10a2f992e33317c169" member_type: VOTER last_known_addr { host: "127.21.51.188" port: 40073 } }
I20260809 16:20:48.793416 32380 leader_election.cc:288] T 00000000000000000000000000000000 P 04e0582b6c844b10a2f992e33317c169 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4896338a36d9400f84bedb07f140d7ad (127.21.51.190:40451), e438fd6b8e6645f7addce3b7676caca0 (127.21.51.189:41439)
I20260809 16:20:48.793978 32201 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "04e0582b6c844b10a2f992e33317c169" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e438fd6b8e6645f7addce3b7676caca0" is_pre_election: true
I20260809 16:20:48.794130 32201 raft_consensus.cc:1799] T 00000000000000000000000000000000 P e438fd6b8e6645f7addce3b7676caca0 [term 1 LEARNER]: Handling vote request from an unknown peer 04e0582b6c844b10a2f992e33317c169
I20260809 16:20:48.794210 32201 raft_consensus.cc:2396] T 00000000000000000000000000000000 P e438fd6b8e6645f7addce3b7676caca0 [term 1 LEARNER]: Leader pre-election vote request: Denying vote to candidate 04e0582b6c844b10a2f992e33317c169 for term 1 because replica has last-logged OpId of term: 1 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 0 index: 0.
I20260809 16:20:48.794536 31277 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "04e0582b6c844b10a2f992e33317c169" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4896338a36d9400f84bedb07f140d7ad" is_pre_election: true
I20260809 16:20:48.794927 32295 leader_election.cc:302] T 00000000000000000000000000000000 P 04e0582b6c844b10a2f992e33317c169 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 04e0582b6c844b10a2f992e33317c169; no voters: 4896338a36d9400f84bedb07f140d7ad, e438fd6b8e6645f7addce3b7676caca0
I20260809 16:20:48.795087 32380 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 04e0582b6c844b10a2f992e33317c169 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:20:48.796005 32380 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 04e0582b6c844b10a2f992e33317c169 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 16:20:48.802361 32331 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 04e0582b6c844b10a2f992e33317c169 [term 1 FOLLOWER]: Refusing update from remote peer 4896338a36d9400f84bedb07f140d7ad: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260809 16:20:48.803881 32414 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 4896338a36d9400f84bedb07f140d7ad [LEADER]: Connected to new peer: Peer: permanent_uuid: "04e0582b6c844b10a2f992e33317c169" member_type: NON_VOTER last_known_addr { host: "127.21.51.188" port: 40073 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.000s
W20260809 16:20:48.804553 31232 consensus_peers.cc:487] T 00000000000000000000000000000000 P 4896338a36d9400f84bedb07f140d7ad -> Peer 04e0582b6c844b10a2f992e33317c169 (127.21.51.188:40073): Couldn't send request to peer 04e0582b6c844b10a2f992e33317c169. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260809 16:20:48.806942 32123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:48.802061 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.189:41439 (local address 127.0.0.1:48414)
0809 16:20:48.802102 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0809 16:20:48.802115 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 16:20:48.802161 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:20:48.803692 (+ 1531us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:20:48.803693 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:20:48.803695 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:20:48.803736 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:20:48.803737 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:20:48.805125 (+ 1388us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:20:48.805126 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:20:48.805876 (+ 750us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:20:48.805878 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:20:48.806694 (+ 816us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:20:48.806696 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:20:48.806756 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:48.806793 (+ 37us) client_negotiation.cc:769] Sending connection context
0809 16:20:48.806835 (+ 42us) client_negotiation.cc:240] Negotiation successful
0809 16:20:48.806882 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":20}
I20260809 16:20:48.807199 32264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:48.802586 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48414 (local address 127.21.51.189:41439)
0809 16:20:48.803493 (+ 907us) server_negotiation.cc:206] Beginning negotiation
0809 16:20:48.803496 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 16:20:48.803506 (+ 10us) server_negotiation.cc:407] Connection header received
0809 16:20:48.803532 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:20:48.803533 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:20:48.803554 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:20:48.803620 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:20:48.803811 (+ 191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:48.804547 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:48.805979 (+ 1432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:48.806529 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:48.806594 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:48.806662 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0809 16:20:48.807096 (+ 434us) server_negotiation.cc:299] Negotiation successful
0809 16:20:48.807139 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":882}
I20260809 16:20:48.807415 32201 raft_consensus.cc:1261] T 00000000000000000000000000000000 P e438fd6b8e6645f7addce3b7676caca0 [term 1 LEARNER]: Refusing update from remote peer 4896338a36d9400f84bedb07f140d7ad: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260809 16:20:48.807725 32414 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 4896338a36d9400f84bedb07f140d7ad [LEADER]: Connected to new peer: Peer: permanent_uuid: "e438fd6b8e6645f7addce3b7676caca0" member_type: NON_VOTER last_known_addr { host: "127.21.51.189" port: 41439 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260809 16:20:48.808293 32201 raft_consensus.cc:2932] T 00000000000000000000000000000000 P e438fd6b8e6645f7addce3b7676caca0 [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 04e0582b6c844b10a2f992e33317c169 (127.21.51.188) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "4896338a36d9400f84bedb07f140d7ad" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 40451 } } peers { permanent_uuid: "e438fd6b8e6645f7addce3b7676caca0" member_type: NON_VOTER last_known_addr { host: "127.21.51.189" port: 41439 } attrs { promote: true } } peers { permanent_uuid: "04e0582b6c844b10a2f992e33317c169" member_type: NON_VOTER last_known_addr { host: "127.21.51.188" port: 40073 } attrs { promote: true } } }
I20260809 16:20:48.809202 21710 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrNqUCn/build/release/bin/kudu
/tmp/dist-test-taskrNqUCn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292417722232-21710-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292417722232-21710-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292417722232-21710-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292417722232-21710-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.51.129:33587
--local_ip_for_outbound_sockets=127.21.51.129
--tserver_master_addrs=127.21.51.190:40451
--webserver_port=39855
--webserver_interface=127.21.51.129
--keytab_file=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292417722232-21710-0/krb5kdc/oryx_127.21.51.129.keytab
--principal=oryx/127.21.51.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.51.148:33645
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.21.51.190:40451,127.21.51.189:41439
--tserver_master_addrs=127.21.51.190:40451,127.21.51.189:41439,127.21.51.188:40073 with env {KRB5CCNAME=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292417722232-21710-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292417722232-21710-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292417722232-21710-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 16:20:48.809211 32270 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 04e0582b6c844b10a2f992e33317c169 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 16:20:48.809288 32270 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 04e0582b6c844b10a2f992e33317c169 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 16:20:48.809327 32270 tablet_replica.cc:330] T 00000000000000000000000000000000 P 04e0582b6c844b10a2f992e33317c169: stopping tablet replica
I20260809 16:20:48.809770 32243 sys_catalog.cc:455] T 00000000000000000000000000000000 P e438fd6b8e6645f7addce3b7676caca0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4896338a36d9400f84bedb07f140d7ad. Latest consensus state: current_term: 1 leader_uuid: "4896338a36d9400f84bedb07f140d7ad" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "4896338a36d9400f84bedb07f140d7ad" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 40451 } } peers { permanent_uuid: "e438fd6b8e6645f7addce3b7676caca0" member_type: NON_VOTER last_known_addr { host: "127.21.51.189" port: 41439 } attrs { promote: true } } peers { permanent_uuid: "04e0582b6c844b10a2f992e33317c169" member_type: NON_VOTER last_known_addr { host: "127.21.51.188" port: 40073 } attrs { promote: true } } }
I20260809 16:20:48.809870 32243 sys_catalog.cc:458] T 00000000000000000000000000000000 P e438fd6b8e6645f7addce3b7676caca0 [sys.catalog]: This master's current role is: LEARNER
I20260809 16:20:48.809937 32243 sys_catalog.cc:455] T 00000000000000000000000000000000 P e438fd6b8e6645f7addce3b7676caca0 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "4896338a36d9400f84bedb07f140d7ad" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "4896338a36d9400f84bedb07f140d7ad" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 40451 } } peers { permanent_uuid: "e438fd6b8e6645f7addce3b7676caca0" member_type: NON_VOTER last_known_addr { host: "127.21.51.189" port: 41439 } attrs { promote: true } } peers { permanent_uuid: "04e0582b6c844b10a2f992e33317c169" member_type: NON_VOTER last_known_addr { host: "127.21.51.188" port: 40073 } attrs { promote: true } } }
I20260809 16:20:48.809993 32243 sys_catalog.cc:458] T 00000000000000000000000000000000 P e438fd6b8e6645f7addce3b7676caca0 [sys.catalog]: This master's current role is: LEARNER
I20260809 16:20:48.810231 32414 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 4896338a36d9400f84bedb07f140d7ad: attempting to promote NON_VOTER e438fd6b8e6645f7addce3b7676caca0 to VOTER
I20260809 16:20:48.810475 32414 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4896338a36d9400f84bedb07f140d7ad [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "4896338a36d9400f84bedb07f140d7ad" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 40451 } } peers { permanent_uuid: "e438fd6b8e6645f7addce3b7676caca0" member_type: VOTER last_known_addr { host: "127.21.51.189" port: 41439 } attrs { promote: false } } peers { permanent_uuid: "04e0582b6c844b10a2f992e33317c169" member_type: NON_VOTER last_known_addr { host: "127.21.51.188" port: 40073 } attrs { promote: true } }
W20260809 16:20:48.811278 31232 consensus_peers.cc:487] T 00000000000000000000000000000000 P 4896338a36d9400f84bedb07f140d7ad -> Peer 04e0582b6c844b10a2f992e33317c169 (127.21.51.188:40073): Couldn't send request to peer 04e0582b6c844b10a2f992e33317c169. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260809 16:20:48.811563 32201 raft_consensus.cc:1261] T 00000000000000000000000000000000 P e438fd6b8e6645f7addce3b7676caca0 [term 1 LEARNER]: Refusing update from remote peer 4896338a36d9400f84bedb07f140d7ad: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260809 16:20:48.811753 32124 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 4896338a36d9400f84bedb07f140d7ad [LEADER]: Connected to new peer: Peer: permanent_uuid: "e438fd6b8e6645f7addce3b7676caca0" member_type: VOTER last_known_addr { host: "127.21.51.189" port: 41439 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260809 16:20:48.812397 32124 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 4896338a36d9400f84bedb07f140d7ad [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, e438fd6b8e6645f7addce3b7676caca0 (127.21.51.189) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "4896338a36d9400f84bedb07f140d7ad" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 40451 } } peers { permanent_uuid: "e438fd6b8e6645f7addce3b7676caca0" member_type: VOTER last_known_addr { host: "127.21.51.189" port: 41439 } attrs { promote: false } } peers { permanent_uuid: "04e0582b6c844b10a2f992e33317c169" member_type: NON_VOTER last_known_addr { host: "127.21.51.188" port: 40073 } attrs { promote: true } } }
I20260809 16:20:48.813482 32124 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4896338a36d9400f84bedb07f140d7ad [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "4896338a36d9400f84bedb07f140d7ad" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "4896338a36d9400f84bedb07f140d7ad" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 40451 } } peers { permanent_uuid: "e438fd6b8e6645f7addce3b7676caca0" member_type: VOTER last_known_addr { host: "127.21.51.189" port: 41439 } attrs { promote: false } } peers { permanent_uuid: "04e0582b6c844b10a2f992e33317c169" member_type: NON_VOTER last_known_addr { host: "127.21.51.188" port: 40073 } attrs { promote: true } } }
I20260809 16:20:48.813584 32124 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4896338a36d9400f84bedb07f140d7ad [sys.catalog]: This master's current role is: LEADER
I20260809 16:20:48.813660 32414 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4896338a36d9400f84bedb07f140d7ad [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "4896338a36d9400f84bedb07f140d7ad" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "4896338a36d9400f84bedb07f140d7ad" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 40451 } } peers { permanent_uuid: "e438fd6b8e6645f7addce3b7676caca0" member_type: VOTER last_known_addr { host: "127.21.51.189" port: 41439 } attrs { promote: false } } peers { permanent_uuid: "04e0582b6c844b10a2f992e33317c169" member_type: NON_VOTER last_known_addr { host: "127.21.51.188" port: 40073 } attrs { promote: true } } }
I20260809 16:20:48.813792 32414 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4896338a36d9400f84bedb07f140d7ad [sys.catalog]: This master's current role is: LEADER
I20260809 16:20:48.813869 32201 raft_consensus.cc:2932] T 00000000000000000000000000000000 P e438fd6b8e6645f7addce3b7676caca0 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, e438fd6b8e6645f7addce3b7676caca0 (127.21.51.189) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "4896338a36d9400f84bedb07f140d7ad" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 40451 } } peers { permanent_uuid: "e438fd6b8e6645f7addce3b7676caca0" member_type: VOTER last_known_addr { host: "127.21.51.189" port: 41439 } attrs { promote: false } } peers { permanent_uuid: "04e0582b6c844b10a2f992e33317c169" member_type: NON_VOTER last_known_addr { host: "127.21.51.188" port: 40073 } attrs { promote: true } } }
I20260809 16:20:48.814956 32416 sys_catalog.cc:455] T 00000000000000000000000000000000 P e438fd6b8e6645f7addce3b7676caca0 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "4896338a36d9400f84bedb07f140d7ad" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "4896338a36d9400f84bedb07f140d7ad" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 40451 } } peers { permanent_uuid: "e438fd6b8e6645f7addce3b7676caca0" member_type: VOTER last_known_addr { host: "127.21.51.189" port: 41439 } attrs { promote: false } } peers { permanent_uuid: "04e0582b6c844b10a2f992e33317c169" member_type: NON_VOTER last_known_addr { host: "127.21.51.188" port: 40073 } attrs { promote: true } } }
I20260809 16:20:48.815054 32416 sys_catalog.cc:458] T 00000000000000000000000000000000 P e438fd6b8e6645f7addce3b7676caca0 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 16:20:48.823338 32270 master.cc:552] Master@127.21.51.188:40073 shutdown complete.
I20260809 16:20:48.823391 32270 master_runner.cc:305] Clearing existing system tablet
I20260809 16:20:48.823529 32270 fs_manager.cc:359] Metadata directory not provided
I20260809 16:20:48.823578 32270 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:20:48.824312 32270 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.001s sys 0.000s
I20260809 16:20:48.824857 32419 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:20:48.824971 32270 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292417722232-21710-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:20:48.825018 32270 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 16:20:48.825039 32270 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292417722232-21710-0/minicluster-data/master-2/data,/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292417722232-21710-0/minicluster-data/master-2/wal
uuid: "04e0582b6c844b10a2f992e33317c169"
format_stamp: "Formatted at 2026-08-09 16:20:48 on dist-test-slave-s11t"
server_key: "05ea5f561327bbb8aafb5007706daf6b"
server_key_iv: "ffa73ae1ea8fa325283b8105c82ecefc"
server_key_version: "encryptionkey@0"
I20260809 16:20:48.825753 32270 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 04e0582b6c844b10a2f992e33317c169: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 16:20:48.827370 32270 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 04e0582b6c844b10a2f992e33317c169: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260809 16:20:48.827451 32270 log.cc:1196] T 00000000000000000000000000000000 P 04e0582b6c844b10a2f992e33317c169: Deleting WAL directory at /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292417722232-21710-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260809 16:20:48.827674 32270 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 04e0582b6c844b10a2f992e33317c169: Deleting consensus metadata
I20260809 16:20:48.827814 32270 master_runner.cc:315] Copying system tablet from 127.21.51.190:40451
I20260809 16:20:48.828452 32270 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 04e0582b6c844b10a2f992e33317c169: tablet copy: Beginning tablet copy session from remote peer at address 127.21.51.190:40451
I20260809 16:20:48.836021 31325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:48.828985 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40126 (local address 127.21.51.190:40451)
0809 16:20:48.829035 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0809 16:20:48.829037 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:20:48.829102 (+ 65us) server_negotiation.cc:407] Connection header received
0809 16:20:48.829187 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:20:48.829188 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:20:48.829200 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:20:48.829269 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:20:48.829642 (+ 373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:48.830442 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:48.831483 (+ 1041us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:48.831764 (+ 281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:48.831837 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:48.833001 (+ 1164us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:20:48.833003 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:20:48.833003 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:20:48.833037 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:20:48.834792 (+ 1755us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:20:48.835187 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:20:48.835188 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:20:48.835188 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:20:48.835248 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:20:48.835472 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:20:48.835473 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:20:48.835473 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:20:48.835602 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:20:48.835668 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0809 16:20:48.835853 (+ 185us) server_negotiation.cc:299] Negotiation successful
0809 16:20:48.835933 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260809 16:20:48.836030 32426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:48.828798 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:40451 (local address 127.0.0.1:40126)
0809 16:20:48.829032 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0809 16:20:48.829044 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 16:20:48.829120 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:20:48.829272 (+ 152us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:20:48.829273 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:20:48.829487 (+ 214us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:20:48.829553 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:20:48.829556 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:20:48.830538 (+ 982us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:20:48.830540 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:20:48.831387 (+ 847us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:20:48.831390 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:20:48.831849 (+ 459us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:20:48.831850 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:20:48.831930 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:48.832467 (+ 537us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:20:48.832473 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:20:48.832909 (+ 436us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:20:48.834887 (+ 1978us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:20:48.834888 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:20:48.834889 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:20:48.835118 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:20:48.835323 (+ 205us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:20:48.835324 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:20:48.835325 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:20:48.835399 (+ 74us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:20:48.835679 (+ 280us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:20:48.835680 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:20:48.835733 (+ 53us) client_negotiation.cc:769] Sending connection context
0809 16:20:48.835815 (+ 82us) client_negotiation.cc:240] Negotiation successful
0809 16:20:48.835934 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":207,"thread_start_us":67,"threads_started":1}
I20260809 16:20:48.836732 31290 tablet_copy_service.cc:140] P 4896338a36d9400f84bedb07f140d7ad: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 04e0582b6c844b10a2f992e33317c169 ({username='oryx', principal='oryx/127.21.51.188@KRBTEST.COM'} at 127.0.0.1:40126)
I20260809 16:20:48.836800 31290 tablet_copy_service.cc:161] P 4896338a36d9400f84bedb07f140d7ad: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 04e0582b6c844b10a2f992e33317c169 at {username='oryx', principal='oryx/127.21.51.188@KRBTEST.COM'} at 127.0.0.1:40126: session id = 04e0582b6c844b10a2f992e33317c169-00000000000000000000000000000000
I20260809 16:20:48.837456 31290 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 4896338a36d9400f84bedb07f140d7ad: Tablet Copy: opened 0 blocks and 1 log segments
I20260809 16:20:48.837951 32270 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260809 16:20:48.839249 32270 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 04e0582b6c844b10a2f992e33317c169: tablet copy: Starting download of 0 data blocks...
I20260809 16:20:48.839404 32270 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 04e0582b6c844b10a2f992e33317c169: tablet copy: Starting download of 1 WAL segments...
I20260809 16:20:48.840194 32270 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 04e0582b6c844b10a2f992e33317c169: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260809 16:20:48.841084 31290 tablet_copy_service.cc:342] P 4896338a36d9400f84bedb07f140d7ad: Request end of tablet copy session 04e0582b6c844b10a2f992e33317c169-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.21.51.188@KRBTEST.COM'} at 127.0.0.1:40126
I20260809 16:20:48.841146 31290 tablet_copy_service.cc:433] P 4896338a36d9400f84bedb07f140d7ad: ending tablet copy session 04e0582b6c844b10a2f992e33317c169-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 04e0582b6c844b10a2f992e33317c169
I20260809 16:20:48.843557 32270 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:20:48.844815 32431 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:20:48.845044 32427 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:20:48.845295 32270 server_base.cc:1034] running on GCE node
W20260809 16:20:48.845398 32429 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:20:48.845559 32270 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:20:48.845707 32270 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:20:48.846802 32270 hybrid_clock.cc:648] HybridClock initialized: now 1786292448846788 us; error 33 us; skew 500 ppm
Aug 09 16:20:48 dist-test-slave-s11t krb5kdc[31184](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292448, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.51.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:20:48.848373 32270 init.cc:377] Logged in from keytab as oryx/127.21.51.188@KRBTEST.COM (short username oryx)
I20260809 16:20:48.849404 32270 webserver.cc:460] Webserver started at http://127.21.51.188:33063/ using document root <none> and password file <none>
I20260809 16:20:48.849557 32270 fs_manager.cc:359] Metadata directory not provided
I20260809 16:20:48.849587 32270 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:20:48.850250 32270 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.001s sys 0.000s
I20260809 16:20:48.850773 32441 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:20:48.850865 32270 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 16:20:48.850894 32270 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292417722232-21710-0/minicluster-data/master-2/data,/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292417722232-21710-0/minicluster-data/master-2/wal
uuid: "04e0582b6c844b10a2f992e33317c169"
format_stamp: "Formatted at 2026-08-09 16:20:48 on dist-test-slave-s11t"
server_key: "05ea5f561327bbb8aafb5007706daf6b"
server_key_iv: "ffa73ae1ea8fa325283b8105c82ecefc"
server_key_version: "encryptionkey@0"
I20260809 16:20:48.850941 32270 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292417722232-21710-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292417722232-21710-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292417722232-21710-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:20:48.863636 32270 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:20:48.871052 32270 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:20:48.874132 32270 rpc_server.cc:307] RPC server started. Bound to: 127.21.51.188:40073
I20260809 16:20:48.874486 32533 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.51.188:40073 every 8 connection(s)
I20260809 16:20:48.874673 32270 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292417722232-21710-0/minicluster-data/master-2/data/info.pb
I20260809 16:20:48.875795 32534 sys_catalog.cc:263] Verifying existing consensus state
I20260809 16:20:48.876598 32534 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 04e0582b6c844b10a2f992e33317c169: Bootstrap starting.
I20260809 16:20:48.879812 32534 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 04e0582b6c844b10a2f992e33317c169: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 16:20:48.880152 32534 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 04e0582b6c844b10a2f992e33317c169: Bootstrap complete.
I20260809 16:20:48.880432 32534 raft_consensus.cc:348] T 00000000000000000000000000000000 P 04e0582b6c844b10a2f992e33317c169 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "4896338a36d9400f84bedb07f140d7ad" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 40451 } } peers { permanent_uuid: "e438fd6b8e6645f7addce3b7676caca0" member_type: VOTER last_known_addr { host: "127.21.51.189" port: 41439 } attrs { promote: false } } peers { permanent_uuid: "04e0582b6c844b10a2f992e33317c169" member_type: NON_VOTER last_known_addr { host: "127.21.51.188" port: 40073 } attrs { promote: true } }
I20260809 16:20:48.880623 32534 raft_consensus.cc:729] T 00000000000000000000000000000000 P 04e0582b6c844b10a2f992e33317c169 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 04e0582b6c844b10a2f992e33317c169, State: Initialized, Role: LEARNER
I20260809 16:20:48.880743 32534 consensus_queue.cc:260] T 00000000000000000000000000000000 P 04e0582b6c844b10a2f992e33317c169 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "4896338a36d9400f84bedb07f140d7ad" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 40451 } } peers { permanent_uuid: "e438fd6b8e6645f7addce3b7676caca0" member_type: VOTER last_known_addr { host: "127.21.51.189" port: 41439 } attrs { promote: false } } peers { permanent_uuid: "04e0582b6c844b10a2f992e33317c169" member_type: NON_VOTER last_known_addr { host: "127.21.51.188" port: 40073 } attrs { promote: true } }
I20260809 16:20:48.881053 32534 sys_catalog.cc:564] T 00000000000000000000000000000000 P 04e0582b6c844b10a2f992e33317c169 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 16:20:48.881042 32537 sys_catalog.cc:455] T 00000000000000000000000000000000 P 04e0582b6c844b10a2f992e33317c169 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "4896338a36d9400f84bedb07f140d7ad" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 40451 } } peers { permanent_uuid: "e438fd6b8e6645f7addce3b7676caca0" member_type: VOTER last_known_addr { host: "127.21.51.189" port: 41439 } attrs { promote: false } } peers { permanent_uuid: "04e0582b6c844b10a2f992e33317c169" member_type: NON_VOTER last_known_addr { host: "127.21.51.188" port: 40073 } attrs { promote: true } } }
I20260809 16:20:48.881192 32537 sys_catalog.cc:458] T 00000000000000000000000000000000 P 04e0582b6c844b10a2f992e33317c169 [sys.catalog]: This master's current role is: LEARNER
I20260809 16:20:48.882845 32564 catalog_manager.cc:1259] Loaded cluster ID: 70b9d0b7af95488b90d1785aca0f4ff6
I20260809 16:20:48.882894 32564 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 04e0582b6c844b10a2f992e33317c169: loading cluster ID for follower catalog manager: success
I20260809 16:20:48.884060 32564 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 04e0582b6c844b10a2f992e33317c169: acquiring CA information for follower catalog manager: success
I20260809 16:20:48.884344 32564 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 04e0582b6c844b10a2f992e33317c169: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260809 16:20:48.928580 32417 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:20:48.928874 32417 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:20:48.928907 32417 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:20:48.930402 32417 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:20:48.930455 32417 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:20:48.930490 32417 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.51.129
I20260809 16:20:48.932359 32417 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.51.148:33645
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292417722232-21710-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292417722232-21710-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292417722232-21710-0/krb5kdc/oryx_127.21.51.129.keytab
--principal=oryx/127.21.51.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.51.129:33587
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292417722232-21710-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.51.129
--webserver_port=39855
--tserver_master_addrs=127.21.51.190:40451,127.21.51.189:41439,127.21.51.188:40073
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32417
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.51.129
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292417722232-21710-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:20:48.932548 32417 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:20:48.932809 32417 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:20:48.937932 32579 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:20:48.937924 32576 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:20:48.938155 32417 server_base.cc:1034] running on GCE node
W20260809 16:20:48.937951 32572 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:20:48.938403 32417 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:20:48.938649 32417 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:20:48.939827 32417 hybrid_clock.cc:648] HybridClock initialized: now 1786292448939797 us; error 40 us; skew 500 ppm
Aug 09 16:20:48 dist-test-slave-s11t krb5kdc[31184](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292448, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.51.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:20:48.942488 32417 init.cc:377] Logged in from keytab as oryx/127.21.51.129@KRBTEST.COM (short username oryx)
I20260809 16:20:48.942886 32417 webserver.cc:460] Webserver started at http://127.21.51.129:39855/ using document root <none> and password file <none>
I20260809 16:20:48.943068 32417 fs_manager.cc:359] Metadata directory not provided
I20260809 16:20:48.943123 32417 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:20:48.944419 32417 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:20:48.945042 32587 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:20:48.945225 32417 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 16:20:48.945322 32417 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292417722232-21710-0/minicluster-data/ts-0/data,/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292417722232-21710-0/minicluster-data/ts-0/wal
uuid: "62e7e0d86ddb442abe06e05b9bce2806"
format_stamp: "Formatted at 2026-08-09 16:20:47 on dist-test-slave-s11t"
server_key: "c11050fed4aa359d9f1ec10c20eb7dde"
server_key_iv: "875c836c1829390054293fa15abcc95e"
server_key_version: "encryptionkey@0"
I20260809 16:20:48.945649 32417 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292417722232-21710-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292417722232-21710-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292417722232-21710-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:20:48.963795 32417 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:20:48.966077 32417 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:20:48.966212 32417 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:20:48.966468 32417 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:20:48.966794 32417 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:20:48.966849 32417 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:20:48.966881 32417 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:20:48.966909 32417 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:20:48.972795 32417 rpc_server.cc:307] RPC server started. Bound to: 127.21.51.129:33587
I20260809 16:20:48.972852 32757 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.51.129:33587 every 8 connection(s)
I20260809 16:20:48.973170 32417 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292417722232-21710-0/minicluster-data/ts-0/data/info.pb
I20260809 16:20:48.976399 21710 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrNqUCn/build/release/bin/kudu as pid 32417
I20260809 16:20:48.976540 21710 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrNqUCn/build/release/bin/kudu with pid 31561
Aug 09 16:20:48 dist-test-slave-s11t krb5kdc[31184](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292448, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.51.129@KRBTEST.COM for oryx/127.21.51.190@KRBTEST.COM
I20260809 16:20:48.985740 31325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:48.974007 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.51.129:41253 (local address 127.21.51.190:40451)
0809 16:20:48.974069 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0809 16:20:48.974071 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:20:48.974299 (+ 228us) server_negotiation.cc:407] Connection header received
0809 16:20:48.975381 (+ 1082us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:20:48.975383 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:20:48.975395 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:20:48.975429 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:20:48.976208 (+ 779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:48.977003 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:48.978643 (+ 1640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:48.978954 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:48.978997 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:48.981975 (+ 2978us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:20:48.981977 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:20:48.981977 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:20:48.981985 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:20:48.983966 (+ 1981us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:20:48.984471 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:20:48.984472 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:20:48.984473 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:20:48.984548 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:20:48.984872 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:20:48.984873 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:20:48.984873 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:20:48.985070 (+ 197us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:20:48.985160 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0809 16:20:48.985465 (+ 305us) server_negotiation.cc:299] Negotiation successful
0809 16:20:48.985574 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260809 16:20:48.985828 303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:48.973948 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:40451 (local address 127.21.51.129:41253)
0809 16:20:48.974144 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0809 16:20:48.974187 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0809 16:20:48.975268 (+ 1081us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:20:48.975525 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:20:48.975530 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:20:48.975936 (+ 406us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:20:48.976093 (+ 157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:20:48.976098 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:20:48.977145 (+ 1047us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:20:48.977145 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:20:48.978502 (+ 1357us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:20:48.978505 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:20:48.979091 (+ 586us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:20:48.979092 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:20:48.979253 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:48.980160 (+ 907us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:20:48.980165 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:20:48.981839 (+ 1674us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:20:48.984073 (+ 2234us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:20:48.984080 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:20:48.984082 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:20:48.984386 (+ 304us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:20:48.984628 (+ 242us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:20:48.984629 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:20:48.984629 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:20:48.984794 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:20:48.985150 (+ 356us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:20:48.985155 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:20:48.985312 (+ 157us) client_negotiation.cc:769] Sending connection context
0809 16:20:48.985409 (+ 97us) client_negotiation.cc:240] Negotiation successful
0809 16:20:48.985575 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":169,"mutex_wait_us":127,"thread_start_us":111,"threads_started":1}
I20260809 16:20:48.986316 32760 heartbeater.cc:347] Connected to a master server at 127.21.51.190:40451
I20260809 16:20:48.986399 32760 heartbeater.cc:464] Registering TS with master...
I20260809 16:20:48.986577 32760 heartbeater.cc:511] Master 127.21.51.190:40451 requested a full tablet report, sending...
I20260809 16:20:48.987001 31258 ts_manager.cc:194] Re-registered known tserver with Master: 62e7e0d86ddb442abe06e05b9bce2806 (127.21.51.129:33587)
I20260809 16:20:48.987457 31258 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.21.51.129@KRBTEST.COM'} at 127.21.51.129:41253
Aug 09 16:20:48 dist-test-slave-s11t krb5kdc[31184](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292448, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.51.129@KRBTEST.COM for oryx/127.21.51.189@KRBTEST.COM
Aug 09 16:20:48 dist-test-slave-s11t krb5kdc[31184](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292448, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.51.129@KRBTEST.COM for oryx/127.21.51.188@KRBTEST.COM
I20260809 16:20:48.994333 32264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:48.974414 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.51.129:53617 (local address 127.21.51.189:41439)
0809 16:20:48.974478 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0809 16:20:48.974480 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:20:48.974490 (+ 10us) server_negotiation.cc:407] Connection header received
0809 16:20:48.975380 (+ 890us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:20:48.975381 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:20:48.975395 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:20:48.975429 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:20:48.976206 (+ 777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:48.977009 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:48.978649 (+ 1640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:48.978954 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:48.978997 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:48.991221 (+ 12224us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:20:48.991223 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:20:48.991224 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:20:48.991233 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:20:48.992970 (+ 1737us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:20:48.993402 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:20:48.993403 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:20:48.993404 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:20:48.993461 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:20:48.993695 (+ 234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:20:48.993696 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:20:48.993696 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:20:48.993857 (+ 161us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:20:48.993909 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0809 16:20:48.994127 (+ 218us) server_negotiation.cc:299] Negotiation successful
0809 16:20:48.994215 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260809 16:20:48.994498 32767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:48.973967 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.51.129:54971 (local address 127.21.51.188:40073)
0809 16:20:48.974091 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0809 16:20:48.974093 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:20:48.974361 (+ 268us) server_negotiation.cc:407] Connection header received
0809 16:20:48.975363 (+ 1002us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:20:48.975364 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:20:48.975379 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:20:48.975449 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:20:48.976193 (+ 744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:48.977398 (+ 1205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:48.978801 (+ 1403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:48.979227 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:48.979300 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:48.991282 (+ 11982us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:20:48.991284 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:20:48.991284 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:20:48.991293 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:20:48.993207 (+ 1914us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:20:48.993602 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:20:48.993603 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:20:48.993603 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:20:48.993663 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:20:48.993880 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:20:48.993882 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:20:48.993882 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:20:48.994032 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:20:48.994104 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0809 16:20:48.994291 (+ 187us) server_negotiation.cc:299] Negotiation successful
0809 16:20:48.994381 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93,"thread_start_us":55,"threads_started":1}
I20260809 16:20:48.994524 302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:48.973948 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.188:40073 (local address 127.21.51.129:54971)
0809 16:20:48.974144 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0809 16:20:48.974187 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0809 16:20:48.975254 (+ 1067us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:20:48.975524 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:20:48.975530 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:20:48.975895 (+ 365us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:20:48.976092 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:20:48.976097 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:20:48.977510 (+ 1413us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:20:48.977511 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:20:48.978678 (+ 1167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:20:48.978682 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:20:48.979365 (+ 683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:20:48.979367 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:20:48.979447 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:48.980021 (+ 574us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:20:48.980027 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:20:48.991158 (+ 11131us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:20:48.993309 (+ 2151us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:20:48.993310 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:20:48.993311 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:20:48.993539 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:20:48.993735 (+ 196us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:20:48.993736 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:20:48.993736 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:20:48.993805 (+ 69us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:20:48.994105 (+ 300us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:20:48.994106 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:20:48.994158 (+ 52us) client_negotiation.cc:769] Sending connection context
0809 16:20:48.994250 (+ 92us) client_negotiation.cc:240] Negotiation successful
0809 16:20:48.994380 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":158,"mutex_wait_us":1,"thread_start_us":104,"threads_started":1}
I20260809 16:20:48.995083 304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:48.974292 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.189:41439 (local address 127.21.51.129:53617)
0809 16:20:48.974442 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0809 16:20:48.974452 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 16:20:48.975276 (+ 824us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:20:48.975530 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:20:48.975531 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:20:48.975926 (+ 395us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:20:48.976093 (+ 167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:20:48.976098 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:20:48.977145 (+ 1047us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:20:48.977145 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:20:48.978502 (+ 1357us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:20:48.978505 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:20:48.979093 (+ 588us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:20:48.979093 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:20:48.979253 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:48.979860 (+ 607us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:20:48.979870 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:20:48.991116 (+ 11246us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:20:48.993064 (+ 1948us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:20:48.993065 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:20:48.993066 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:20:48.993306 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:20:48.993546 (+ 240us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:20:48.993546 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:20:48.993547 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:20:48.993612 (+ 65us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:20:48.993967 (+ 355us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:20:48.993967 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:20:48.994007 (+ 40us) client_negotiation.cc:769] Sending connection context
0809 16:20:48.994795 (+ 788us) client_negotiation.cc:240] Negotiation successful
0809 16:20:48.994921 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"mutex_wait_us":55,"thread_start_us":68,"threads_started":1}
I20260809 16:20:48.995117 32759 heartbeater.cc:347] Connected to a master server at 127.21.51.188:40073
I20260809 16:20:48.995307 32759 heartbeater.cc:464] Registering TS with master...
I20260809 16:20:48.995370 21710 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrNqUCn/build/release/bin/kudu
/tmp/dist-test-taskrNqUCn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292417722232-21710-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292417722232-21710-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292417722232-21710-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292417722232-21710-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.51.130:34781
--local_ip_for_outbound_sockets=127.21.51.130
--tserver_master_addrs=127.21.51.190:40451
--webserver_port=35549
--webserver_interface=127.21.51.130
--keytab_file=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292417722232-21710-0/krb5kdc/oryx_127.21.51.130.keytab
--principal=oryx/127.21.51.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.51.148:33645
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.21.51.190:40451,127.21.51.189:41439
--tserver_master_addrs=127.21.51.190:40451,127.21.51.189:41439,127.21.51.188:40073 with env {KRB5CCNAME=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292417722232-21710-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292417722232-21710-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292417722232-21710-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 16:20:48.995419 32759 heartbeater.cc:511] Master 127.21.51.188:40073 requested a full tablet report, sending...
I20260809 16:20:48.995795 32465 ts_manager.cc:194] Registered new tserver with Master: 62e7e0d86ddb442abe06e05b9bce2806 (127.21.51.129:33587)
I20260809 16:20:48.995886 32762 heartbeater.cc:347] Connected to a master server at 127.21.51.189:41439
I20260809 16:20:48.995942 32762 heartbeater.cc:464] Registering TS with master...
I20260809 16:20:48.995994 32762 heartbeater.cc:511] Master 127.21.51.189:41439 requested a full tablet report, sending...
I20260809 16:20:48.996372 32187 ts_manager.cc:194] Registered new tserver with Master: 62e7e0d86ddb442abe06e05b9bce2806 (127.21.51.129:33587)
W20260809 16:20:49.079595 305 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:20:49.079790 305 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:20:49.079813 305 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:20:49.081039 305 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:20:49.081096 305 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:20:49.081142 305 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.51.130
I20260809 16:20:49.082888 305 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.51.148:33645
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292417722232-21710-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292417722232-21710-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292417722232-21710-0/krb5kdc/oryx_127.21.51.130.keytab
--principal=oryx/127.21.51.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.51.130:34781
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292417722232-21710-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.51.130
--webserver_port=35549
--tserver_master_addrs=127.21.51.190:40451,127.21.51.189:41439,127.21.51.188:40073
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.305
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.51.130
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292417722232-21710-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:20:49.083073 305 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:20:49.083248 305 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:20:49.088286 315 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:20:49.088338 317 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:20:49.088457 305 server_base.cc:1034] running on GCE node
W20260809 16:20:49.088338 313 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:20:49.088811 305 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:20:49.089108 305 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:20:49.090273 305 hybrid_clock.cc:648] HybridClock initialized: now 1786292449090253 us; error 33 us; skew 500 ppm
Aug 09 16:20:49 dist-test-slave-s11t krb5kdc[31184](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292449, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.51.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:20:49.092758 305 init.cc:377] Logged in from keytab as oryx/127.21.51.130@KRBTEST.COM (short username oryx)
I20260809 16:20:49.093210 305 webserver.cc:460] Webserver started at http://127.21.51.130:35549/ using document root <none> and password file <none>
I20260809 16:20:49.093457 305 fs_manager.cc:359] Metadata directory not provided
I20260809 16:20:49.093541 305 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:20:49.094854 305 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:20:49.095521 328 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:20:49.095695 305 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:20:49.095800 305 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292417722232-21710-0/minicluster-data/ts-1/data,/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292417722232-21710-0/minicluster-data/ts-1/wal
uuid: "d3af45dba9b04300879578698c6a2922"
format_stamp: "Formatted at 2026-08-09 16:20:48 on dist-test-slave-s11t"
server_key: "6da352be3b2a44ddc55026aaf1bdd162"
server_key_iv: "a7e2578e84826695c298e53cce96c8dc"
server_key_version: "encryptionkey@0"
I20260809 16:20:49.096153 305 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292417722232-21710-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292417722232-21710-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292417722232-21710-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:20:49.115582 305 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:20:49.117771 305 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:20:49.117925 305 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:20:49.118160 305 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:20:49.118474 305 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:20:49.118507 305 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:20:49.118530 305 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:20:49.118543 305 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:20:49.124333 305 rpc_server.cc:307] RPC server started. Bound to: 127.21.51.130:34781
I20260809 16:20:49.124367 491 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.51.130:34781 every 8 connection(s)
I20260809 16:20:49.124728 305 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292417722232-21710-0/minicluster-data/ts-1/data/info.pb
I20260809 16:20:49.130993 21710 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrNqUCn/build/release/bin/kudu as pid 305
I20260809 16:20:49.131170 21710 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrNqUCn/build/release/bin/kudu with pid 31776
Aug 09 16:20:49 dist-test-slave-s11t krb5kdc[31184](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292449, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.51.130@KRBTEST.COM for oryx/127.21.51.188@KRBTEST.COM
Aug 09 16:20:49 dist-test-slave-s11t krb5kdc[31184](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292449, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.51.130@KRBTEST.COM for oryx/127.21.51.189@KRBTEST.COM
Aug 09 16:20:49 dist-test-slave-s11t krb5kdc[31184](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292449, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.51.130@KRBTEST.COM for oryx/127.21.51.190@KRBTEST.COM
I20260809 16:20:49.137353 32767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:49.125537 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.51.130:34465 (local address 127.21.51.188:40073)
0809 16:20:49.125591 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0809 16:20:49.125593 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:20:49.125803 (+ 210us) server_negotiation.cc:407] Connection header received
0809 16:20:49.126397 (+ 594us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:20:49.126399 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:20:49.126411 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:20:49.126458 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:20:49.127485 (+ 1027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:49.127999 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:49.129119 (+ 1120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:49.129335 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:49.129418 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:49.132995 (+ 3577us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:20:49.132997 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:20:49.132997 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:20:49.133010 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:20:49.134574 (+ 1564us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:20:49.135226 (+ 652us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:20:49.135228 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:20:49.135228 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:20:49.135290 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:20:49.136084 (+ 794us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:20:49.136085 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:20:49.136085 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:20:49.136642 (+ 557us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:20:49.136720 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0809 16:20:49.137120 (+ 400us) server_negotiation.cc:299] Negotiation successful
0809 16:20:49.137215 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260809 16:20:49.137962 501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:49.125486 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.188:40073 (local address 127.21.51.130:34465)
0809 16:20:49.125720 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0809 16:20:49.125728 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0809 16:20:49.126294 (+ 566us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:20:49.126821 (+ 527us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:20:49.126821 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:20:49.127113 (+ 292us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:20:49.127390 (+ 277us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:20:49.127395 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:20:49.128111 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:20:49.128112 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:20:49.129028 (+ 916us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:20:49.129031 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:20:49.129481 (+ 450us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:20:49.129482 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:20:49.129615 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:49.130145 (+ 530us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:20:49.130155 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:20:49.132896 (+ 2741us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:20:49.134840 (+ 1944us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:20:49.134845 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:20:49.134848 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:20:49.135143 (+ 295us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:20:49.135380 (+ 237us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:20:49.135381 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:20:49.135381 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:20:49.135987 (+ 606us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:20:49.136778 (+ 791us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:20:49.136785 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:20:49.136948 (+ 163us) client_negotiation.cc:769] Sending connection context
0809 16:20:49.137495 (+ 547us) client_negotiation.cc:240] Negotiation successful
0809 16:20:49.137712 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":212,"mutex_wait_us":20,"spinlock_wait_cycles":8448,"thread_start_us":99,"threads_started":1}
I20260809 16:20:49.138496 31325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:49.125451 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.51.130:59219 (local address 127.21.51.190:40451)
0809 16:20:49.125503 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0809 16:20:49.125506 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 16:20:49.125924 (+ 418us) server_negotiation.cc:407] Connection header received
0809 16:20:49.126418 (+ 494us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:20:49.126419 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:20:49.126431 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:20:49.126465 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:20:49.127505 (+ 1040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:49.128261 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:49.129815 (+ 1554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:49.130066 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:49.130137 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:49.135426 (+ 5289us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:20:49.135428 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:20:49.135428 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:20:49.135436 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:20:49.137239 (+ 1803us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:20:49.137636 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:20:49.137638 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:20:49.137638 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:20:49.137701 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:20:49.137914 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:20:49.137915 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:20:49.137916 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:20:49.138049 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:20:49.138094 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0809 16:20:49.138311 (+ 217us) server_negotiation.cc:299] Negotiation successful
0809 16:20:49.138400 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260809 16:20:49.138496 492 heartbeater.cc:347] Connected to a master server at 127.21.51.188:40073
I20260809 16:20:49.138589 492 heartbeater.cc:464] Registering TS with master...
I20260809 16:20:49.138790 492 heartbeater.cc:511] Master 127.21.51.188:40073 requested a full tablet report, sending...
I20260809 16:20:49.138878 500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:49.125481 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:40451 (local address 127.21.51.130:59219)
0809 16:20:49.125646 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0809 16:20:49.125686 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0809 16:20:49.126290 (+ 604us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:20:49.126549 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:20:49.126554 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:20:49.127078 (+ 524us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:20:49.127391 (+ 313us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:20:49.127397 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:20:49.128364 (+ 967us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:20:49.128364 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:20:49.129305 (+ 941us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:20:49.129307 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:20:49.130150 (+ 843us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:20:49.130151 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:20:49.130239 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:49.131025 (+ 786us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:20:49.131030 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:20:49.135328 (+ 4298us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:20:49.137321 (+ 1993us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:20:49.137322 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:20:49.137322 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:20:49.137547 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:20:49.137777 (+ 230us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:20:49.137778 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:20:49.137778 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:20:49.137844 (+ 66us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:20:49.138145 (+ 301us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:20:49.138146 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:20:49.138176 (+ 30us) client_negotiation.cc:769] Sending connection context
0809 16:20:49.138620 (+ 444us) client_negotiation.cc:240] Negotiation successful
0809 16:20:49.138727 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"mutex_wait_us":78,"thread_start_us":87,"threads_started":1}
I20260809 16:20:49.138994 503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:49.125769 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.189:41439 (local address 127.21.51.130:60749)
0809 16:20:49.125947 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0809 16:20:49.125957 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 16:20:49.126302 (+ 345us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:20:49.126553 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:20:49.126556 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:20:49.127090 (+ 534us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:20:49.127391 (+ 301us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:20:49.127397 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:20:49.128363 (+ 966us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:20:49.128363 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:20:49.129309 (+ 946us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:20:49.129310 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:20:49.129758 (+ 448us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:20:49.129759 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:20:49.129839 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:49.130426 (+ 587us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:20:49.130431 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:20:49.135019 (+ 4588us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:20:49.137274 (+ 2255us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:20:49.137275 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:20:49.137276 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:20:49.137782 (+ 506us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:20:49.138254 (+ 472us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:20:49.138254 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:20:49.138255 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:20:49.138326 (+ 71us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:20:49.138648 (+ 322us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:20:49.138649 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:20:49.138692 (+ 43us) client_negotiation.cc:769] Sending connection context
0809 16:20:49.138785 (+ 93us) client_negotiation.cc:240] Negotiation successful
0809 16:20:49.138902 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"mutex_wait_us":69,"thread_start_us":85,"threads_started":1}
I20260809 16:20:49.139027 32264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:49.125887 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.51.130:60749 (local address 127.21.51.189:41439)
0809 16:20:49.125945 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0809 16:20:49.125947 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:20:49.125999 (+ 52us) server_negotiation.cc:407] Connection header received
0809 16:20:49.126417 (+ 418us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:20:49.126418 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:20:49.126431 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:20:49.126466 (+ 35us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:20:49.127503 (+ 1037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:49.128264 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:49.129398 (+ 1134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:49.129673 (+ 275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:49.129755 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:49.135168 (+ 5413us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:20:49.135170 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:20:49.135171 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:20:49.135180 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:20:49.137180 (+ 2000us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:20:49.137882 (+ 702us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:20:49.137884 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:20:49.137884 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:20:49.137947 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:20:49.138407 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:20:49.138408 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:20:49.138409 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:20:49.138549 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:20:49.138595 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0809 16:20:49.138827 (+ 232us) server_negotiation.cc:299] Negotiation successful
0809 16:20:49.138912 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260809 16:20:49.139412 494 heartbeater.cc:347] Connected to a master server at 127.21.51.190:40451
I20260809 16:20:49.139470 494 heartbeater.cc:464] Registering TS with master...
I20260809 16:20:49.139541 494 heartbeater.cc:511] Master 127.21.51.190:40451 requested a full tablet report, sending...
I20260809 16:20:49.139853 496 heartbeater.cc:347] Connected to a master server at 127.21.51.189:41439
I20260809 16:20:49.139921 496 heartbeater.cc:464] Registering TS with master...
I20260809 16:20:49.139911 31258 ts_manager.cc:194] Re-registered known tserver with Master: d3af45dba9b04300879578698c6a2922 (127.21.51.130:34781)
I20260809 16:20:49.139997 496 heartbeater.cc:511] Master 127.21.51.189:41439 requested a full tablet report, sending...
I20260809 16:20:49.140429 31258 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.21.51.130@KRBTEST.COM'} at 127.21.51.130:59219
I20260809 16:20:49.140980 32187 ts_manager.cc:194] Registered new tserver with Master: d3af45dba9b04300879578698c6a2922 (127.21.51.130:34781)
I20260809 16:20:49.141245 32465 ts_manager.cc:194] Registered new tserver with Master: d3af45dba9b04300879578698c6a2922 (127.21.51.130:34781)
I20260809 16:20:49.145440 21710 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrNqUCn/build/release/bin/kudu
/tmp/dist-test-taskrNqUCn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292417722232-21710-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292417722232-21710-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292417722232-21710-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292417722232-21710-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.51.131:46809
--local_ip_for_outbound_sockets=127.21.51.131
--tserver_master_addrs=127.21.51.190:40451
--webserver_port=33963
--webserver_interface=127.21.51.131
--keytab_file=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292417722232-21710-0/krb5kdc/oryx_127.21.51.131.keytab
--principal=oryx/127.21.51.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.51.148:33645
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.21.51.190:40451,127.21.51.189:41439
--tserver_master_addrs=127.21.51.190:40451,127.21.51.189:41439,127.21.51.188:40073 with env {KRB5CCNAME=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292417722232-21710-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292417722232-21710-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292417722232-21710-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:20:49.228725 504 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:20:49.228950 504 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:20:49.228988 504 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:20:49.230108 504 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:20:49.230171 504 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:20:49.230216 504 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.51.131
I20260809 16:20:49.231658 504 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.51.148:33645
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292417722232-21710-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292417722232-21710-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292417722232-21710-0/krb5kdc/oryx_127.21.51.131.keytab
--principal=oryx/127.21.51.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.51.131:46809
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292417722232-21710-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.51.131
--webserver_port=33963
--tserver_master_addrs=127.21.51.190:40451,127.21.51.189:41439,127.21.51.188:40073
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.504
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.51.131
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292417722232-21710-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:20:49.231927 504 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:20:49.232156 504 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:20:49.237528 509 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:20:49.237594 514 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:20:49.237528 511 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:20:49.237735 504 server_base.cc:1034] running on GCE node
I20260809 16:20:49.237937 504 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:20:49.238176 504 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:20:49.239339 504 hybrid_clock.cc:648] HybridClock initialized: now 1786292449239316 us; error 50 us; skew 500 ppm
Aug 09 16:20:49 dist-test-slave-s11t krb5kdc[31184](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292449, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.51.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:20:49.241890 504 init.cc:377] Logged in from keytab as oryx/127.21.51.131@KRBTEST.COM (short username oryx)
I20260809 16:20:49.242308 504 webserver.cc:460] Webserver started at http://127.21.51.131:33963/ using document root <none> and password file <none>
I20260809 16:20:49.242558 504 fs_manager.cc:359] Metadata directory not provided
I20260809 16:20:49.242640 504 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:20:49.244031 504 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:20:49.244747 524 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:20:49.244918 504 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:20:49.245005 504 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292417722232-21710-0/minicluster-data/ts-2/data,/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292417722232-21710-0/minicluster-data/ts-2/wal
uuid: "2bf53ab985e048d19a823bea592a496d"
format_stamp: "Formatted at 2026-08-09 16:20:48 on dist-test-slave-s11t"
server_key: "bb0534026278ef00034db15d6b7da243"
server_key_iv: "7c749655fa21bf7b9c100a0407aeaec5"
server_key_version: "encryptionkey@0"
I20260809 16:20:49.245333 504 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292417722232-21710-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292417722232-21710-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292417722232-21710-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:20:49.279462 504 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:20:49.281981 504 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:20:49.282137 504 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:20:49.282394 504 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:20:49.282724 504 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:20:49.282764 504 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:20:49.282815 504 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:20:49.282840 504 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:20:49.288622 504 rpc_server.cc:307] RPC server started. Bound to: 127.21.51.131:46809
I20260809 16:20:49.288724 689 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.51.131:46809 every 8 connection(s)
I20260809 16:20:49.289088 504 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292417722232-21710-0/minicluster-data/ts-2/data/info.pb
I20260809 16:20:49.291033 21710 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrNqUCn/build/release/bin/kudu as pid 504
Aug 09 16:20:49 dist-test-slave-s11t krb5kdc[31184](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292449, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.51.131@KRBTEST.COM for oryx/127.21.51.188@KRBTEST.COM
Aug 09 16:20:49 dist-test-slave-s11t krb5kdc[31184](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292449, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.51.131@KRBTEST.COM for oryx/127.21.51.190@KRBTEST.COM
Aug 09 16:20:49 dist-test-slave-s11t krb5kdc[31184](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292449, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.51.131@KRBTEST.COM for oryx/127.21.51.189@KRBTEST.COM
I20260809 16:20:49.302942 718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:49.293041 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48422 (local address 127.21.51.189:41439)
0809 16:20:49.294150 (+ 1109us) server_negotiation.cc:206] Beginning negotiation
0809 16:20:49.294152 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:20:49.294159 (+ 7us) server_negotiation.cc:407] Connection header received
0809 16:20:49.294178 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:20:49.294179 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:20:49.294189 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:20:49.294247 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:20:49.294916 (+ 669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:49.295655 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:49.296670 (+ 1015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:49.297026 (+ 356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:49.297095 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:49.299012 (+ 1917us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:20:49.299013 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:20:49.299014 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:20:49.299021 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:20:49.301109 (+ 2088us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:20:49.301681 (+ 572us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:20:49.301682 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:20:49.301683 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:20:49.301738 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:20:49.301959 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:20:49.301960 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:20:49.301960 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:20:49.302099 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:20:49.302194 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0809 16:20:49.302610 (+ 416us) server_negotiation.cc:299] Negotiation successful
0809 16:20:49.302724 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1089,"thread_start_us":102,"threads_started":1}
I20260809 16:20:49.303045 699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:49.289790 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.188:40073 (local address 127.21.51.131:38429)
0809 16:20:49.290052 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0809 16:20:49.290062 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 16:20:49.290653 (+ 591us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:20:49.291315 (+ 662us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:20:49.291317 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:20:49.291632 (+ 315us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:20:49.291738 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:20:49.291740 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:20:49.292757 (+ 1017us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:20:49.292758 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:20:49.294863 (+ 2105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:20:49.294865 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:20:49.295377 (+ 512us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:20:49.295378 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:20:49.295520 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:49.296076 (+ 556us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:20:49.296086 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:20:49.297773 (+ 1687us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:20:49.301303 (+ 3530us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:20:49.301309 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:20:49.301312 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:20:49.301625 (+ 313us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:20:49.301860 (+ 235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:20:49.301861 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:20:49.301861 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:20:49.301984 (+ 123us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:20:49.302293 (+ 309us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:20:49.302299 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:20:49.302456 (+ 157us) client_negotiation.cc:769] Sending connection context
0809 16:20:49.302585 (+ 129us) client_negotiation.cc:240] Negotiation successful
0809 16:20:49.302811 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":240,"mutex_wait_us":1143,"thread_start_us":117,"threads_started":1}
I20260809 16:20:49.303364 32767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:49.289870 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.51.131:38429 (local address 127.21.51.188:40073)
0809 16:20:49.289934 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0809 16:20:49.289936 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:20:49.290123 (+ 187us) server_negotiation.cc:407] Connection header received
0809 16:20:49.290748 (+ 625us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:20:49.290749 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:20:49.290761 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:20:49.290832 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:20:49.291850 (+ 1018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:49.292608 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:49.294954 (+ 2346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:49.295243 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:49.295315 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:49.298078 (+ 2763us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:20:49.298080 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:20:49.298080 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:20:49.298087 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:20:49.301203 (+ 3116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:20:49.301712 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:20:49.301713 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:20:49.301714 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:20:49.301788 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:20:49.302059 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:20:49.302060 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:20:49.302060 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:20:49.302196 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:20:49.302262 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0809 16:20:49.303176 (+ 914us) server_negotiation.cc:299] Negotiation successful
0809 16:20:49.303259 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260809 16:20:49.303715 691 heartbeater.cc:347] Connected to a master server at 127.21.51.188:40073
I20260809 16:20:49.303819 691 heartbeater.cc:464] Registering TS with master...
I20260809 16:20:49.304016 709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:49.292162 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44186 (local address 127.21.51.188:40073)
0809 16:20:49.293209 (+ 1047us) server_negotiation.cc:206] Beginning negotiation
0809 16:20:49.293211 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:20:49.293327 (+ 116us) server_negotiation.cc:407] Connection header received
0809 16:20:49.293349 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:20:49.293350 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:20:49.293362 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:20:49.293420 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:20:49.294102 (+ 682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:49.294883 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:49.295981 (+ 1098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:49.296287 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:49.296598 (+ 311us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:49.298625 (+ 2027us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:20:49.298626 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:20:49.298627 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:20:49.298635 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:20:49.301525 (+ 2890us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:20:49.302028 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:20:49.302029 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:20:49.302029 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:20:49.302078 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:20:49.302293 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:20:49.302294 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:20:49.302294 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:20:49.302430 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:20:49.303801 (+ 1371us) server_negotiation.cc:1035] Waiting for connection context
0809 16:20:49.303845 (+ 44us) server_negotiation.cc:299] Negotiation successful
0809 16:20:49.303925 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1028,"thread_start_us":56,"threads_started":1}
I20260809 16:20:49.304021 691 heartbeater.cc:511] Master 127.21.51.188:40073 requested a full tablet report, sending...
I20260809 16:20:49.304397 32465 ts_manager.cc:194] Registered new tserver with Master: 2bf53ab985e048d19a823bea592a496d (127.21.51.131:46809)
I20260809 16:20:49.304709 714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:49.292523 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40132 (local address 127.21.51.190:40451)
0809 16:20:49.293415 (+ 892us) server_negotiation.cc:206] Beginning negotiation
0809 16:20:49.293417 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:20:49.293424 (+ 7us) server_negotiation.cc:407] Connection header received
0809 16:20:49.293445 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:20:49.293446 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:20:49.293459 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:20:49.293533 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:20:49.294232 (+ 699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:49.294961 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:49.296184 (+ 1223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:49.296489 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:49.296554 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:49.298979 (+ 2425us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:20:49.298980 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:20:49.298981 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:20:49.298988 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:20:49.301261 (+ 2273us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:20:49.302509 (+ 1248us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:20:49.302510 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:20:49.302510 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:20:49.302599 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:20:49.302823 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:20:49.302824 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:20:49.302825 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:20:49.302953 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:20:49.303057 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0809 16:20:49.304467 (+ 1410us) server_negotiation.cc:299] Negotiation successful
0809 16:20:49.304593 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":864,"thread_start_us":75,"threads_started":1}
I20260809 16:20:49.306317 31325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:49.289983 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.51.131:41415 (local address 127.21.51.190:40451)
0809 16:20:49.290037 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0809 16:20:49.290039 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:20:49.290096 (+ 57us) server_negotiation.cc:407] Connection header received
0809 16:20:49.290786 (+ 690us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:20:49.290787 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:20:49.290801 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:20:49.290833 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:20:49.291842 (+ 1009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:49.292613 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:49.298617 (+ 6004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:49.298957 (+ 340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:49.299002 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:49.303202 (+ 4200us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:20:49.303204 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:20:49.303204 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:20:49.303210 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:20:49.305010 (+ 1800us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:20:49.305438 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:20:49.305440 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:20:49.305440 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:20:49.305502 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:20:49.305721 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:20:49.305722 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:20:49.305723 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:20:49.305861 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:20:49.305934 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0809 16:20:49.306147 (+ 213us) server_negotiation.cc:299] Negotiation successful
0809 16:20:49.306233 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260809 16:20:49.306664 698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:49.289787 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:40451 (local address 127.21.51.131:41415)
0809 16:20:49.289972 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0809 16:20:49.290014 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0809 16:20:49.290653 (+ 639us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:20:49.290913 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:20:49.290918 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:20:49.291420 (+ 502us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:20:49.291726 (+ 306us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:20:49.291731 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:20:49.292767 (+ 1036us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:20:49.292767 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:20:49.298513 (+ 5746us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:20:49.298518 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:20:49.299105 (+ 587us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:20:49.299105 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:20:49.299188 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:49.300038 (+ 850us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:20:49.300043 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:20:49.301742 (+ 1699us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:20:49.305080 (+ 3338us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:20:49.305081 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:20:49.305082 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:20:49.305338 (+ 256us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:20:49.305575 (+ 237us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:20:49.305576 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:20:49.305576 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:20:49.305634 (+ 58us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:20:49.305966 (+ 332us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:20:49.305967 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:20:49.306010 (+ 43us) client_negotiation.cc:769] Sending connection context
0809 16:20:49.306456 (+ 446us) client_negotiation.cc:240] Negotiation successful
0809 16:20:49.306563 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"mutex_wait_us":1123,"thread_start_us":111,"threads_started":1}
I20260809 16:20:49.307137 694 heartbeater.cc:347] Connected to a master server at 127.21.51.190:40451
I20260809 16:20:49.307188 694 heartbeater.cc:464] Registering TS with master...
I20260809 16:20:49.307201 32264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:49.290173 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.51.131:48563 (local address 127.21.51.189:41439)
0809 16:20:49.290219 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0809 16:20:49.290220 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 16:20:49.290319 (+ 99us) server_negotiation.cc:407] Connection header received
0809 16:20:49.290778 (+ 459us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:20:49.290779 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:20:49.290792 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:20:49.290828 (+ 36us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:20:49.291840 (+ 1012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:49.292614 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:49.299376 (+ 6762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:49.299688 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:49.299730 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:49.304019 (+ 4289us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:20:49.304020 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:20:49.304021 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:20:49.304028 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:20:49.305830 (+ 1802us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:20:49.306350 (+ 520us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:20:49.306351 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:20:49.306352 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:20:49.306437 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:20:49.306662 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:20:49.306663 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:20:49.306664 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:20:49.306813 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:20:49.306863 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0809 16:20:49.307050 (+ 187us) server_negotiation.cc:299] Negotiation successful
0809 16:20:49.307124 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260809 16:20:49.307255 694 heartbeater.cc:511] Master 127.21.51.190:40451 requested a full tablet report, sending...
I20260809 16:20:49.307497 31259 ts_manager.cc:194] Re-registered known tserver with Master: 2bf53ab985e048d19a823bea592a496d (127.21.51.131:46809)
I20260809 16:20:49.307862 701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:49.290101 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.189:41439 (local address 127.21.51.131:48563)
0809 16:20:49.290242 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0809 16:20:49.290250 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0809 16:20:49.290654 (+ 404us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:20:49.290925 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:20:49.290925 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:20:49.291440 (+ 515us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:20:49.291726 (+ 286us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:20:49.291731 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:20:49.292758 (+ 1027us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:20:49.292759 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:20:49.294012 (+ 1253us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:20:49.294015 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:20:49.299806 (+ 5791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:20:49.299806 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:20:49.299875 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:49.301368 (+ 1493us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:20:49.301373 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:20:49.303094 (+ 1721us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:20:49.305901 (+ 2807us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:20:49.305902 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:20:49.305902 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:20:49.306120 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:20:49.306525 (+ 405us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:20:49.306526 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:20:49.306526 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:20:49.306585 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:20:49.306912 (+ 327us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:20:49.306913 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:20:49.306952 (+ 39us) client_negotiation.cc:769] Sending connection context
0809 16:20:49.307018 (+ 66us) client_negotiation.cc:240] Negotiation successful
0809 16:20:49.307292 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"mutex_wait_us":26,"thread_start_us":59,"threads_started":1}
I20260809 16:20:49.308012 31259 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.21.51.131@KRBTEST.COM'} at 127.21.51.131:41415
I20260809 16:20:49.308506 696 heartbeater.cc:347] Connected to a master server at 127.21.51.189:41439
I20260809 16:20:49.308558 696 heartbeater.cc:464] Registering TS with master...
I20260809 16:20:49.306871 31258 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40132:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 16:20:49.308604 696 heartbeater.cc:511] Master 127.21.51.189:41439 requested a full tablet report, sending...
W20260809 16:20:49.308785 31258 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 16:20:49.308887 32187 ts_manager.cc:194] Registered new tserver with Master: 2bf53ab985e048d19a823bea592a496d (127.21.51.131:46809)
I20260809 16:20:49.316352 727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:49.312484 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38212 (local address 127.21.51.131:46809)
0809 16:20:49.312622 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0809 16:20:49.312626 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 16:20:49.312814 (+ 188us) server_negotiation.cc:407] Connection header received
0809 16:20:49.312934 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:20:49.312936 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:20:49.312952 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:20:49.313006 (+ 54us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:20:49.313233 (+ 227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:49.314324 (+ 1091us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:49.315463 (+ 1139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:49.315992 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:49.316071 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:49.316146 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0809 16:20:49.316221 (+ 75us) server_negotiation.cc:299] Negotiation successful
0809 16:20:49.316277 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":109,"thread_start_us":82,"threads_started":1}
I20260809 16:20:49.316354 723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:49.312315 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.131:46809 (local address 127.0.0.1:38212)
0809 16:20:49.312739 (+ 424us) negotiation.cc:107] Waiting for socket to connect
0809 16:20:49.312750 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 16:20:49.312843 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:20:49.313103 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:20:49.313104 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:20:49.313106 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:20:49.313171 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:20:49.313174 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:20:49.314416 (+ 1242us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:20:49.314416 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:20:49.315379 (+ 963us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:20:49.315381 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:20:49.316072 (+ 691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:20:49.316073 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:20:49.316144 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:49.316149 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 16:20:49.316209 (+ 60us) client_negotiation.cc:240] Negotiation successful
0809 16:20:49.316276 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":411,"mutex_wait_us":4,"thread_start_us":92,"threads_started":1}
I20260809 16:20:49.316903 725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:49.312297 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.130:34781 (local address 127.0.0.1:43182)
0809 16:20:49.312467 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0809 16:20:49.312478 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 16:20:49.312531 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:20:49.313272 (+ 741us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:20:49.313272 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:20:49.313274 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:20:49.313314 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:20:49.313315 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:20:49.314506 (+ 1191us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:20:49.314507 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:20:49.315644 (+ 1137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:20:49.315647 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:20:49.316609 (+ 962us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:20:49.316610 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:20:49.316678 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:49.316717 (+ 39us) client_negotiation.cc:769] Sending connection context
0809 16:20:49.316787 (+ 70us) client_negotiation.cc:240] Negotiation successful
0809 16:20:49.316832 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":91,"threads_started":1}
I20260809 16:20:49.316936 728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:49.312529 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43182 (local address 127.21.51.130:34781)
0809 16:20:49.313132 (+ 603us) server_negotiation.cc:206] Beginning negotiation
0809 16:20:49.313137 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 16:20:49.313147 (+ 10us) server_negotiation.cc:407] Connection header received
0809 16:20:49.313184 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:20:49.313186 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:20:49.313200 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:20:49.313282 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:20:49.313362 (+ 80us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:49.314402 (+ 1040us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:49.315741 (+ 1339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:49.316526 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:49.316600 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:49.316661 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0809 16:20:49.316808 (+ 147us) server_negotiation.cc:299] Negotiation successful
0809 16:20:49.316860 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":567,"thread_start_us":96,"threads_started":1}
I20260809 16:20:49.317404 599 tablet_service.cc:1467] Processing CreateTablet for tablet 6bc338c4bb5940b7a0ed142751987835 (DEFAULT_TABLE table=test-table [id=029d9aa0fc464fc6924deae45fd7b8c2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:20:49.317741 599 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6bc338c4bb5940b7a0ed142751987835. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:20:49.317692 398 tablet_service.cc:1467] Processing CreateTablet for tablet 6bc338c4bb5940b7a0ed142751987835 (DEFAULT_TABLE table=test-table [id=029d9aa0fc464fc6924deae45fd7b8c2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:20:49.317917 724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:49.312294 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.129:33587 (local address 127.0.0.1:44924)
0809 16:20:49.312433 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0809 16:20:49.312445 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 16:20:49.312502 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:20:49.313106 (+ 604us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:20:49.313107 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:20:49.313108 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:20:49.313171 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:20:49.313173 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:20:49.315671 (+ 2498us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:20:49.315671 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:20:49.316792 (+ 1121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:20:49.316796 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:20:49.317689 (+ 893us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:20:49.317691 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:20:49.317756 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:49.317761 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 16:20:49.317793 (+ 32us) client_negotiation.cc:240] Negotiation successful
0809 16:20:49.317831 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"mutex_wait_us":11,"thread_start_us":59,"threads_started":1}
I20260809 16:20:49.318037 398 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6bc338c4bb5940b7a0ed142751987835. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:20:49.318301 731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:49.312642 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44924 (local address 127.21.51.129:33587)
0809 16:20:49.312931 (+ 289us) server_negotiation.cc:206] Beginning negotiation
0809 16:20:49.312935 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 16:20:49.312946 (+ 11us) server_negotiation.cc:407] Connection header received
0809 16:20:49.312989 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:20:49.312991 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:20:49.313009 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:20:49.313102 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:20:49.314429 (+ 1327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:49.315565 (+ 1136us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:49.316892 (+ 1327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:49.317580 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:49.318081 (+ 501us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:49.318165 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0809 16:20:49.318191 (+ 26us) server_negotiation.cc:299] Negotiation successful
0809 16:20:49.318227 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":251,"thread_start_us":97,"threads_started":1}
I20260809 16:20:49.318969 32664 tablet_service.cc:1467] Processing CreateTablet for tablet 6bc338c4bb5940b7a0ed142751987835 (DEFAULT_TABLE table=test-table [id=029d9aa0fc464fc6924deae45fd7b8c2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:20:49.319290 32664 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6bc338c4bb5940b7a0ed142751987835. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:20:49.320662 732 tablet_bootstrap.cc:492] T 6bc338c4bb5940b7a0ed142751987835 P d3af45dba9b04300879578698c6a2922: Bootstrap starting.
I20260809 16:20:49.321372 732 tablet_bootstrap.cc:654] T 6bc338c4bb5940b7a0ed142751987835 P d3af45dba9b04300879578698c6a2922: Neither blocks nor log segments found. Creating new log.
I20260809 16:20:49.321704 732 log.cc:824] T 6bc338c4bb5940b7a0ed142751987835 P d3af45dba9b04300879578698c6a2922: Log is configured to *not* fsync() on all Append() calls
I20260809 16:20:49.322027 734 tablet_bootstrap.cc:492] T 6bc338c4bb5940b7a0ed142751987835 P 2bf53ab985e048d19a823bea592a496d: Bootstrap starting.
I20260809 16:20:49.322562 732 tablet_bootstrap.cc:492] T 6bc338c4bb5940b7a0ed142751987835 P d3af45dba9b04300879578698c6a2922: No bootstrap required, opened a new log
I20260809 16:20:49.322638 737 tablet_bootstrap.cc:492] T 6bc338c4bb5940b7a0ed142751987835 P 62e7e0d86ddb442abe06e05b9bce2806: Bootstrap starting.
I20260809 16:20:49.322762 732 ts_tablet_manager.cc:1397] T 6bc338c4bb5940b7a0ed142751987835 P d3af45dba9b04300879578698c6a2922: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260809 16:20:49.322810 734 tablet_bootstrap.cc:654] T 6bc338c4bb5940b7a0ed142751987835 P 2bf53ab985e048d19a823bea592a496d: Neither blocks nor log segments found. Creating new log.
I20260809 16:20:49.323138 734 log.cc:824] T 6bc338c4bb5940b7a0ed142751987835 P 2bf53ab985e048d19a823bea592a496d: Log is configured to *not* fsync() on all Append() calls
I20260809 16:20:49.323221 737 tablet_bootstrap.cc:654] T 6bc338c4bb5940b7a0ed142751987835 P 62e7e0d86ddb442abe06e05b9bce2806: Neither blocks nor log segments found. Creating new log.
I20260809 16:20:49.323563 737 log.cc:824] T 6bc338c4bb5940b7a0ed142751987835 P 62e7e0d86ddb442abe06e05b9bce2806: Log is configured to *not* fsync() on all Append() calls
I20260809 16:20:49.323917 734 tablet_bootstrap.cc:492] T 6bc338c4bb5940b7a0ed142751987835 P 2bf53ab985e048d19a823bea592a496d: No bootstrap required, opened a new log
I20260809 16:20:49.323989 734 ts_tablet_manager.cc:1397] T 6bc338c4bb5940b7a0ed142751987835 P 2bf53ab985e048d19a823bea592a496d: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 16:20:49.324445 737 tablet_bootstrap.cc:492] T 6bc338c4bb5940b7a0ed142751987835 P 62e7e0d86ddb442abe06e05b9bce2806: No bootstrap required, opened a new log
I20260809 16:20:49.324538 737 ts_tablet_manager.cc:1397] T 6bc338c4bb5940b7a0ed142751987835 P 62e7e0d86ddb442abe06e05b9bce2806: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 16:20:49.325567 732 raft_consensus.cc:348] T 6bc338c4bb5940b7a0ed142751987835 P d3af45dba9b04300879578698c6a2922 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62e7e0d86ddb442abe06e05b9bce2806" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 33587 } } peers { permanent_uuid: "d3af45dba9b04300879578698c6a2922" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 34781 } } peers { permanent_uuid: "2bf53ab985e048d19a823bea592a496d" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 46809 } }
I20260809 16:20:49.325734 732 raft_consensus.cc:374] T 6bc338c4bb5940b7a0ed142751987835 P d3af45dba9b04300879578698c6a2922 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:20:49.325769 732 raft_consensus.cc:729] T 6bc338c4bb5940b7a0ed142751987835 P d3af45dba9b04300879578698c6a2922 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d3af45dba9b04300879578698c6a2922, State: Initialized, Role: FOLLOWER
I20260809 16:20:49.325899 732 consensus_queue.cc:260] T 6bc338c4bb5940b7a0ed142751987835 P d3af45dba9b04300879578698c6a2922 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62e7e0d86ddb442abe06e05b9bce2806" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 33587 } } peers { permanent_uuid: "d3af45dba9b04300879578698c6a2922" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 34781 } } peers { permanent_uuid: "2bf53ab985e048d19a823bea592a496d" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 46809 } }
I20260809 16:20:49.326304 732 ts_tablet_manager.cc:1428] T 6bc338c4bb5940b7a0ed142751987835 P d3af45dba9b04300879578698c6a2922: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260809 16:20:49.326493 494 heartbeater.cc:503] Master 127.21.51.190:40451 was elected leader, sending a full tablet report...
I20260809 16:20:49.326821 737 raft_consensus.cc:348] T 6bc338c4bb5940b7a0ed142751987835 P 62e7e0d86ddb442abe06e05b9bce2806 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62e7e0d86ddb442abe06e05b9bce2806" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 33587 } } peers { permanent_uuid: "d3af45dba9b04300879578698c6a2922" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 34781 } } peers { permanent_uuid: "2bf53ab985e048d19a823bea592a496d" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 46809 } }
I20260809 16:20:49.326967 737 raft_consensus.cc:374] T 6bc338c4bb5940b7a0ed142751987835 P 62e7e0d86ddb442abe06e05b9bce2806 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:20:49.327000 737 raft_consensus.cc:729] T 6bc338c4bb5940b7a0ed142751987835 P 62e7e0d86ddb442abe06e05b9bce2806 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 62e7e0d86ddb442abe06e05b9bce2806, State: Initialized, Role: FOLLOWER
I20260809 16:20:49.327114 737 consensus_queue.cc:260] T 6bc338c4bb5940b7a0ed142751987835 P 62e7e0d86ddb442abe06e05b9bce2806 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62e7e0d86ddb442abe06e05b9bce2806" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 33587 } } peers { permanent_uuid: "d3af45dba9b04300879578698c6a2922" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 34781 } } peers { permanent_uuid: "2bf53ab985e048d19a823bea592a496d" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 46809 } }
I20260809 16:20:49.327450 737 ts_tablet_manager.cc:1428] T 6bc338c4bb5940b7a0ed142751987835 P 62e7e0d86ddb442abe06e05b9bce2806: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 16:20:49.327494 32760 heartbeater.cc:503] Master 127.21.51.190:40451 was elected leader, sending a full tablet report...
I20260809 16:20:49.327404 734 raft_consensus.cc:348] T 6bc338c4bb5940b7a0ed142751987835 P 2bf53ab985e048d19a823bea592a496d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62e7e0d86ddb442abe06e05b9bce2806" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 33587 } } peers { permanent_uuid: "d3af45dba9b04300879578698c6a2922" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 34781 } } peers { permanent_uuid: "2bf53ab985e048d19a823bea592a496d" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 46809 } }
I20260809 16:20:49.328408 734 raft_consensus.cc:374] T 6bc338c4bb5940b7a0ed142751987835 P 2bf53ab985e048d19a823bea592a496d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:20:49.328462 734 raft_consensus.cc:729] T 6bc338c4bb5940b7a0ed142751987835 P 2bf53ab985e048d19a823bea592a496d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2bf53ab985e048d19a823bea592a496d, State: Initialized, Role: FOLLOWER
I20260809 16:20:49.328588 734 consensus_queue.cc:260] T 6bc338c4bb5940b7a0ed142751987835 P 2bf53ab985e048d19a823bea592a496d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62e7e0d86ddb442abe06e05b9bce2806" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 33587 } } peers { permanent_uuid: "d3af45dba9b04300879578698c6a2922" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 34781 } } peers { permanent_uuid: "2bf53ab985e048d19a823bea592a496d" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 46809 } }
I20260809 16:20:49.328868 734 ts_tablet_manager.cc:1428] T 6bc338c4bb5940b7a0ed142751987835 P 2bf53ab985e048d19a823bea592a496d: Time spent starting tablet: real 0.005s user 0.003s sys 0.000s
I20260809 16:20:49.328967 694 heartbeater.cc:503] Master 127.21.51.190:40451 was elected leader, sending a full tablet report...
I20260809 16:20:49.355149 709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:49.352663 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44188 (local address 127.21.51.188:40073)
0809 16:20:49.352762 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0809 16:20:49.352764 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:20:49.352773 (+ 9us) server_negotiation.cc:407] Connection header received
0809 16:20:49.352837 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:20:49.352837 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:20:49.352856 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:20:49.352916 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:20:49.353048 (+ 132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:49.353544 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:49.354497 (+ 953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:49.354876 (+ 379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:49.354934 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:49.354983 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0809 16:20:49.355068 (+ 85us) server_negotiation.cc:299] Negotiation successful
0809 16:20:49.355094 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260809 16:20:49.355165 724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:49.352582 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.188:40073 (local address 127.0.0.1:44188)
0809 16:20:49.352654 (+ 72us) negotiation.cc:107] Waiting for socket to connect
0809 16:20:49.352667 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 16:20:49.352745 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:20:49.352933 (+ 188us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:20:49.352934 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:20:49.352935 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:20:49.352983 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:20:49.352985 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:20:49.353606 (+ 621us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:20:49.353607 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:20:49.354416 (+ 809us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:20:49.354418 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:20:49.354956 (+ 538us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:20:49.354956 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:20:49.355012 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:49.355015 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 16:20:49.355064 (+ 49us) client_negotiation.cc:240] Negotiation successful
0809 16:20:49.355106 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":45}
I20260809 16:20:49.355589 32479 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 04e0582b6c844b10a2f992e33317c169 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.9] Dedup: 1.7->[1.8-1.9]
I20260809 16:20:49.356528 32537 sys_catalog.cc:455] T 00000000000000000000000000000000 P 04e0582b6c844b10a2f992e33317c169 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4896338a36d9400f84bedb07f140d7ad. Latest consensus state: current_term: 1 leader_uuid: "4896338a36d9400f84bedb07f140d7ad" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "4896338a36d9400f84bedb07f140d7ad" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 40451 } } peers { permanent_uuid: "e438fd6b8e6645f7addce3b7676caca0" member_type: VOTER last_known_addr { host: "127.21.51.189" port: 41439 } attrs { promote: false } } peers { permanent_uuid: "04e0582b6c844b10a2f992e33317c169" member_type: NON_VOTER last_known_addr { host: "127.21.51.188" port: 40073 } attrs { promote: true } } }
I20260809 16:20:49.356633 32537 sys_catalog.cc:458] T 00000000000000000000000000000000 P 04e0582b6c844b10a2f992e33317c169 [sys.catalog]: This master's current role is: LEARNER
W20260809 16:20:49.375566 497 tablet.cc:2369] T 6bc338c4bb5940b7a0ed142751987835 P d3af45dba9b04300879578698c6a2922: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 16:20:49.474152 32764 tablet.cc:2369] T 6bc338c4bb5940b7a0ed142751987835 P 62e7e0d86ddb442abe06e05b9bce2806: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 16:20:49.539920 697 tablet.cc:2369] T 6bc338c4bb5940b7a0ed142751987835 P 2bf53ab985e048d19a823bea592a496d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 16:20:49.607601 746 raft_consensus.cc:482] T 6bc338c4bb5940b7a0ed142751987835 P 2bf53ab985e048d19a823bea592a496d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 16:20:49.607736 746 raft_consensus.cc:504] T 6bc338c4bb5940b7a0ed142751987835 P 2bf53ab985e048d19a823bea592a496d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62e7e0d86ddb442abe06e05b9bce2806" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 33587 } } peers { permanent_uuid: "d3af45dba9b04300879578698c6a2922" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 34781 } } peers { permanent_uuid: "2bf53ab985e048d19a823bea592a496d" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 46809 } }
I20260809 16:20:49.608067 746 leader_election.cc:288] T 6bc338c4bb5940b7a0ed142751987835 P 2bf53ab985e048d19a823bea592a496d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 62e7e0d86ddb442abe06e05b9bce2806 (127.21.51.129:33587), d3af45dba9b04300879578698c6a2922 (127.21.51.130:34781)
I20260809 16:20:49.611224 728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:49.608407 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.51.131:53351 (local address 127.21.51.130:34781)
0809 16:20:49.608484 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0809 16:20:49.608486 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:20:49.608538 (+ 52us) server_negotiation.cc:407] Connection header received
0809 16:20:49.608620 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:20:49.608621 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:20:49.608632 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:20:49.608737 (+ 105us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:20:49.608933 (+ 196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:49.609404 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:49.610445 (+ 1041us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:49.610842 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:49.610945 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:49.610955 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0809 16:20:49.611116 (+ 161us) server_negotiation.cc:299] Negotiation successful
0809 16:20:49.611159 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260809 16:20:49.611224 698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:49.608265 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.130:34781 (local address 127.21.51.131:53351)
0809 16:20:49.608486 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0809 16:20:49.608497 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 16:20:49.608552 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:20:49.608760 (+ 208us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:20:49.608761 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:20:49.608763 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:20:49.608833 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:20:49.608834 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:20:49.609493 (+ 659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:20:49.609494 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:20:49.610317 (+ 823us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:20:49.610319 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:20:49.610949 (+ 630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:20:49.610949 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:20:49.611039 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:49.611043 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 16:20:49.611110 (+ 67us) client_negotiation.cc:240] Negotiation successful
0809 16:20:49.611149 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":207,"mutex_wait_us":173}
I20260809 16:20:49.611460 701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:49.608264 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.129:33587 (local address 127.21.51.131:38619)
0809 16:20:49.608315 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0809 16:20:49.608330 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 16:20:49.608397 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:20:49.608589 (+ 192us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:20:49.608590 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:20:49.608593 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:20:49.608714 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:20:49.608716 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:20:49.609461 (+ 745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:20:49.609462 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:20:49.610350 (+ 888us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:20:49.610353 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:20:49.611140 (+ 787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:20:49.611141 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:20:49.611231 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:49.611234 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 16:20:49.611304 (+ 70us) client_negotiation.cc:240] Negotiation successful
0809 16:20:49.611355 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":25,"mutex_wait_us":61}
I20260809 16:20:49.611594 432 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6bc338c4bb5940b7a0ed142751987835" candidate_uuid: "2bf53ab985e048d19a823bea592a496d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3af45dba9b04300879578698c6a2922" is_pre_election: true
I20260809 16:20:49.611770 432 raft_consensus.cc:2454] T 6bc338c4bb5940b7a0ed142751987835 P d3af45dba9b04300879578698c6a2922 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2bf53ab985e048d19a823bea592a496d in term 0.
I20260809 16:20:49.611876 731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:49.608361 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.51.131:38619 (local address 127.21.51.129:33587)
0809 16:20:49.608448 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0809 16:20:49.608451 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 16:20:49.608462 (+ 11us) server_negotiation.cc:407] Connection header received
0809 16:20:49.608496 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:20:49.608497 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:20:49.608508 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:20:49.608566 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:20:49.608808 (+ 242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:49.609353 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:49.610454 (+ 1101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:49.611030 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:49.611141 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:49.611150 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0809 16:20:49.611753 (+ 603us) server_negotiation.cc:299] Negotiation successful
0809 16:20:49.611800 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260809 16:20:49.612069 525 leader_election.cc:302] T 6bc338c4bb5940b7a0ed142751987835 P 2bf53ab985e048d19a823bea592a496d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2bf53ab985e048d19a823bea592a496d, d3af45dba9b04300879578698c6a2922; no voters:
I20260809 16:20:49.612145 32690 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6bc338c4bb5940b7a0ed142751987835" candidate_uuid: "2bf53ab985e048d19a823bea592a496d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "62e7e0d86ddb442abe06e05b9bce2806" is_pre_election: true
I20260809 16:20:49.612231 746 raft_consensus.cc:2781] T 6bc338c4bb5940b7a0ed142751987835 P 2bf53ab985e048d19a823bea592a496d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 16:20:49.612286 746 raft_consensus.cc:482] T 6bc338c4bb5940b7a0ed142751987835 P 2bf53ab985e048d19a823bea592a496d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 16:20:49.612303 32690 raft_consensus.cc:2454] T 6bc338c4bb5940b7a0ed142751987835 P 62e7e0d86ddb442abe06e05b9bce2806 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2bf53ab985e048d19a823bea592a496d in term 0.
I20260809 16:20:49.612319 746 raft_consensus.cc:3037] T 6bc338c4bb5940b7a0ed142751987835 P 2bf53ab985e048d19a823bea592a496d [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:20:49.613150 746 raft_consensus.cc:504] T 6bc338c4bb5940b7a0ed142751987835 P 2bf53ab985e048d19a823bea592a496d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62e7e0d86ddb442abe06e05b9bce2806" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 33587 } } peers { permanent_uuid: "d3af45dba9b04300879578698c6a2922" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 34781 } } peers { permanent_uuid: "2bf53ab985e048d19a823bea592a496d" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 46809 } }
I20260809 16:20:49.613306 746 leader_election.cc:288] T 6bc338c4bb5940b7a0ed142751987835 P 2bf53ab985e048d19a823bea592a496d [CANDIDATE]: Term 1 election: Requested vote from peers 62e7e0d86ddb442abe06e05b9bce2806 (127.21.51.129:33587), d3af45dba9b04300879578698c6a2922 (127.21.51.130:34781)
I20260809 16:20:49.613467 32690 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6bc338c4bb5940b7a0ed142751987835" candidate_uuid: "2bf53ab985e048d19a823bea592a496d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "62e7e0d86ddb442abe06e05b9bce2806"
I20260809 16:20:49.613487 432 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6bc338c4bb5940b7a0ed142751987835" candidate_uuid: "2bf53ab985e048d19a823bea592a496d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3af45dba9b04300879578698c6a2922"
I20260809 16:20:49.613540 32690 raft_consensus.cc:3037] T 6bc338c4bb5940b7a0ed142751987835 P 62e7e0d86ddb442abe06e05b9bce2806 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:20:49.613613 432 raft_consensus.cc:3037] T 6bc338c4bb5940b7a0ed142751987835 P d3af45dba9b04300879578698c6a2922 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:20:49.614279 32690 raft_consensus.cc:2454] T 6bc338c4bb5940b7a0ed142751987835 P 62e7e0d86ddb442abe06e05b9bce2806 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2bf53ab985e048d19a823bea592a496d in term 1.
I20260809 16:20:49.614287 432 raft_consensus.cc:2454] T 6bc338c4bb5940b7a0ed142751987835 P d3af45dba9b04300879578698c6a2922 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2bf53ab985e048d19a823bea592a496d in term 1.
I20260809 16:20:49.614471 538 leader_election.cc:302] T 6bc338c4bb5940b7a0ed142751987835 P 2bf53ab985e048d19a823bea592a496d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2bf53ab985e048d19a823bea592a496d, 62e7e0d86ddb442abe06e05b9bce2806; no voters:
I20260809 16:20:49.614568 746 raft_consensus.cc:2781] T 6bc338c4bb5940b7a0ed142751987835 P 2bf53ab985e048d19a823bea592a496d [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:20:49.614713 746 raft_consensus.cc:686] T 6bc338c4bb5940b7a0ed142751987835 P 2bf53ab985e048d19a823bea592a496d [term 1 LEADER]: Becoming Leader. State: Replica: 2bf53ab985e048d19a823bea592a496d, State: Running, Role: LEADER
I20260809 16:20:49.614806 746 consensus_queue.cc:237] T 6bc338c4bb5940b7a0ed142751987835 P 2bf53ab985e048d19a823bea592a496d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62e7e0d86ddb442abe06e05b9bce2806" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 33587 } } peers { permanent_uuid: "d3af45dba9b04300879578698c6a2922" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 34781 } } peers { permanent_uuid: "2bf53ab985e048d19a823bea592a496d" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 46809 } }
I20260809 16:20:49.615833 31258 catalog_manager.cc:5495] T 6bc338c4bb5940b7a0ed142751987835 P 2bf53ab985e048d19a823bea592a496d reported cstate change: term changed from 0 to 1, leader changed from <none> to 2bf53ab985e048d19a823bea592a496d (127.21.51.131). New cstate: current_term: 1 leader_uuid: "2bf53ab985e048d19a823bea592a496d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62e7e0d86ddb442abe06e05b9bce2806" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 33587 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d3af45dba9b04300879578698c6a2922" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 34781 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2bf53ab985e048d19a823bea592a496d" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 46809 } health_report { overall_health: HEALTHY } } }
I20260809 16:20:49.616411 32414 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4896338a36d9400f84bedb07f140d7ad [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "4896338a36d9400f84bedb07f140d7ad" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "4896338a36d9400f84bedb07f140d7ad" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 40451 } } peers { permanent_uuid: "e438fd6b8e6645f7addce3b7676caca0" member_type: VOTER last_known_addr { host: "127.21.51.189" port: 41439 } attrs { promote: false } } peers { permanent_uuid: "04e0582b6c844b10a2f992e33317c169" member_type: NON_VOTER last_known_addr { host: "127.21.51.188" port: 40073 } attrs { promote: true } } }
I20260809 16:20:49.616509 32414 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4896338a36d9400f84bedb07f140d7ad [sys.catalog]: This master's current role is: LEADER
I20260809 16:20:49.617158 755 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 4896338a36d9400f84bedb07f140d7ad: attempting to promote NON_VOTER 04e0582b6c844b10a2f992e33317c169 to VOTER
I20260809 16:20:49.617318 755 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4896338a36d9400f84bedb07f140d7ad [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 1.10, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "4896338a36d9400f84bedb07f140d7ad" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 40451 } } peers { permanent_uuid: "e438fd6b8e6645f7addce3b7676caca0" member_type: VOTER last_known_addr { host: "127.21.51.189" port: 41439 } attrs { promote: false } } peers { permanent_uuid: "04e0582b6c844b10a2f992e33317c169" member_type: VOTER last_known_addr { host: "127.21.51.188" port: 40073 } attrs { promote: false } }
I20260809 16:20:49.617797 32201 raft_consensus.cc:1261] T 00000000000000000000000000000000 P e438fd6b8e6645f7addce3b7676caca0 [term 1 FOLLOWER]: Refusing update from remote peer 4896338a36d9400f84bedb07f140d7ad: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20260809 16:20:49.617831 32478 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 04e0582b6c844b10a2f992e33317c169 [term 1 LEARNER]: Refusing update from remote peer 4896338a36d9400f84bedb07f140d7ad: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20260809 16:20:49.618034 32414 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 4896338a36d9400f84bedb07f140d7ad [LEADER]: Connected to new peer: Peer: permanent_uuid: "e438fd6b8e6645f7addce3b7676caca0" member_type: VOTER last_known_addr { host: "127.21.51.189" port: 41439 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260809 16:20:49.618160 32125 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 4896338a36d9400f84bedb07f140d7ad [LEADER]: Connected to new peer: Peer: permanent_uuid: "04e0582b6c844b10a2f992e33317c169" member_type: VOTER last_known_addr { host: "127.21.51.188" port: 40073 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260809 16:20:49.618683 32414 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 4896338a36d9400f84bedb07f140d7ad [term 1 LEADER]: Committing config change with OpId 1.11: config changed from index 7 to 11, 04e0582b6c844b10a2f992e33317c169 (127.21.51.188) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "4896338a36d9400f84bedb07f140d7ad" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 40451 } } peers { permanent_uuid: "e438fd6b8e6645f7addce3b7676caca0" member_type: VOTER last_known_addr { host: "127.21.51.189" port: 41439 } attrs { promote: false } } peers { permanent_uuid: "04e0582b6c844b10a2f992e33317c169" member_type: VOTER last_known_addr { host: "127.21.51.188" port: 40073 } attrs { promote: false } } }
I20260809 16:20:49.619107 32478 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 04e0582b6c844b10a2f992e33317c169 [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 7 to 11, 04e0582b6c844b10a2f992e33317c169 (127.21.51.188) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "4896338a36d9400f84bedb07f140d7ad" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 40451 } } peers { permanent_uuid: "e438fd6b8e6645f7addce3b7676caca0" member_type: VOTER last_known_addr { host: "127.21.51.189" port: 41439 } attrs { promote: false } } peers { permanent_uuid: "04e0582b6c844b10a2f992e33317c169" member_type: VOTER last_known_addr { host: "127.21.51.188" port: 40073 } attrs { promote: false } } }
I20260809 16:20:49.619738 32414 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4896338a36d9400f84bedb07f140d7ad [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "4896338a36d9400f84bedb07f140d7ad" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "4896338a36d9400f84bedb07f140d7ad" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 40451 } } peers { permanent_uuid: "e438fd6b8e6645f7addce3b7676caca0" member_type: VOTER last_known_addr { host: "127.21.51.189" port: 41439 } attrs { promote: false } } peers { permanent_uuid: "04e0582b6c844b10a2f992e33317c169" member_type: VOTER last_known_addr { host: "127.21.51.188" port: 40073 } attrs { promote: false } } }
I20260809 16:20:49.619822 32414 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4896338a36d9400f84bedb07f140d7ad [sys.catalog]: This master's current role is: LEADER
I20260809 16:20:49.619880 32414 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4896338a36d9400f84bedb07f140d7ad [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "4896338a36d9400f84bedb07f140d7ad" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "4896338a36d9400f84bedb07f140d7ad" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 40451 } } peers { permanent_uuid: "e438fd6b8e6645f7addce3b7676caca0" member_type: VOTER last_known_addr { host: "127.21.51.189" port: 41439 } attrs { promote: false } } peers { permanent_uuid: "04e0582b6c844b10a2f992e33317c169" member_type: VOTER last_known_addr { host: "127.21.51.188" port: 40073 } attrs { promote: false } } }
I20260809 16:20:49.619937 32414 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4896338a36d9400f84bedb07f140d7ad [sys.catalog]: This master's current role is: LEADER
I20260809 16:20:49.619980 32414 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4896338a36d9400f84bedb07f140d7ad [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "4896338a36d9400f84bedb07f140d7ad" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "4896338a36d9400f84bedb07f140d7ad" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 40451 } } peers { permanent_uuid: "e438fd6b8e6645f7addce3b7676caca0" member_type: VOTER last_known_addr { host: "127.21.51.189" port: 41439 } attrs { promote: false } } peers { permanent_uuid: "04e0582b6c844b10a2f992e33317c169" member_type: VOTER last_known_addr { host: "127.21.51.188" port: 40073 } attrs { promote: false } } }
I20260809 16:20:49.620021 32414 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4896338a36d9400f84bedb07f140d7ad [sys.catalog]: This master's current role is: LEADER
I20260809 16:20:49.620230 32201 raft_consensus.cc:2932] T 00000000000000000000000000000000 P e438fd6b8e6645f7addce3b7676caca0 [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 7 to 11, 04e0582b6c844b10a2f992e33317c169 (127.21.51.188) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "4896338a36d9400f84bedb07f140d7ad" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 40451 } } peers { permanent_uuid: "e438fd6b8e6645f7addce3b7676caca0" member_type: VOTER last_known_addr { host: "127.21.51.189" port: 41439 } attrs { promote: false } } peers { permanent_uuid: "04e0582b6c844b10a2f992e33317c169" member_type: VOTER last_known_addr { host: "127.21.51.188" port: 40073 } attrs { promote: false } } }
I20260809 16:20:49.620289 32537 sys_catalog.cc:455] T 00000000000000000000000000000000 P 04e0582b6c844b10a2f992e33317c169 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "4896338a36d9400f84bedb07f140d7ad" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "4896338a36d9400f84bedb07f140d7ad" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 40451 } } peers { permanent_uuid: "e438fd6b8e6645f7addce3b7676caca0" member_type: VOTER last_known_addr { host: "127.21.51.189" port: 41439 } attrs { promote: false } } peers { permanent_uuid: "04e0582b6c844b10a2f992e33317c169" member_type: VOTER last_known_addr { host: "127.21.51.188" port: 40073 } attrs { promote: false } } }
I20260809 16:20:49.620358 32537 sys_catalog.cc:458] T 00000000000000000000000000000000 P 04e0582b6c844b10a2f992e33317c169 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 16:20:49.621189 758 sys_catalog.cc:455] T 00000000000000000000000000000000 P e438fd6b8e6645f7addce3b7676caca0 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "4896338a36d9400f84bedb07f140d7ad" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "4896338a36d9400f84bedb07f140d7ad" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 40451 } } peers { permanent_uuid: "e438fd6b8e6645f7addce3b7676caca0" member_type: VOTER last_known_addr { host: "127.21.51.189" port: 41439 } attrs { promote: false } } peers { permanent_uuid: "04e0582b6c844b10a2f992e33317c169" member_type: VOTER last_known_addr { host: "127.21.51.188" port: 40073 } attrs { promote: false } } }
I20260809 16:20:49.621321 758 sys_catalog.cc:458] T 00000000000000000000000000000000 P e438fd6b8e6645f7addce3b7676caca0 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 16:20:49.672128 727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:49.669551 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38216 (local address 127.21.51.131:46809)
0809 16:20:49.669624 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0809 16:20:49.669626 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:20:49.669634 (+ 8us) server_negotiation.cc:407] Connection header received
0809 16:20:49.669682 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:20:49.669682 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:20:49.669702 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:20:49.669775 (+ 73us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 16:20:49.669949 (+ 174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:49.670535 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:49.671297 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:49.671513 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:49.671591 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:49.671783 (+ 192us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 16:20:49.671902 (+ 119us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 16:20:49.672001 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0809 16:20:49.672020 (+ 19us) server_negotiation.cc:299] Negotiation successful
0809 16:20:49.672040 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260809 16:20:49.674001 432 raft_consensus.cc:1261] T 6bc338c4bb5940b7a0ed142751987835 P d3af45dba9b04300879578698c6a2922 [term 1 FOLLOWER]: Refusing update from remote peer 2bf53ab985e048d19a823bea592a496d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 16:20:49.674018 32690 raft_consensus.cc:1261] T 6bc338c4bb5940b7a0ed142751987835 P 62e7e0d86ddb442abe06e05b9bce2806 [term 1 FOLLOWER]: Refusing update from remote peer 2bf53ab985e048d19a823bea592a496d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 16:20:49.674396 746 consensus_queue.cc:1035] T 6bc338c4bb5940b7a0ed142751987835 P 2bf53ab985e048d19a823bea592a496d [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3af45dba9b04300879578698c6a2922" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 34781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 16:20:49.674543 753 consensus_queue.cc:1035] T 6bc338c4bb5940b7a0ed142751987835 P 2bf53ab985e048d19a823bea592a496d [LEADER]: Connected to new peer: Peer: permanent_uuid: "62e7e0d86ddb442abe06e05b9bce2806" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 33587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 16:20:49.677654 764 mvcc.cc:205] Tried to move back new op lower bound from 7316653873861980160 to 7316653873622753280. Current Snapshot: MvccSnapshot[applied={T|T < 7316653873861980160 or (T in {7316653873861980160})}]
I20260809 16:20:49.680394 31259 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40132:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 16:20:49.680511 31259 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40132:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 16:20:49.682148 31259 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4896338a36d9400f84bedb07f140d7ad: Sending DeleteTablet for 3 replicas of tablet 6bc338c4bb5940b7a0ed142751987835
I20260809 16:20:49.682446 398 tablet_service.cc:1514] Processing DeleteTablet for tablet 6bc338c4bb5940b7a0ed142751987835 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:20:49 UTC) from {username='oryx', principal='oryx/127.21.51.190@KRBTEST.COM'} at 127.0.0.1:43182
I20260809 16:20:49.682618 32664 tablet_service.cc:1514] Processing DeleteTablet for tablet 6bc338c4bb5940b7a0ed142751987835 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:20:49 UTC) from {username='oryx', principal='oryx/127.21.51.190@KRBTEST.COM'} at 127.0.0.1:44924
I20260809 16:20:49.682760 778 tablet_replica.cc:330] T 6bc338c4bb5940b7a0ed142751987835 P d3af45dba9b04300879578698c6a2922: stopping tablet replica
I20260809 16:20:49.682864 778 raft_consensus.cc:2229] T 6bc338c4bb5940b7a0ed142751987835 P d3af45dba9b04300879578698c6a2922 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 16:20:49.682883 599 tablet_service.cc:1514] Processing DeleteTablet for tablet 6bc338c4bb5940b7a0ed142751987835 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:20:49 UTC) from {username='oryx', principal='oryx/127.21.51.190@KRBTEST.COM'} at 127.0.0.1:38212
I20260809 16:20:49.682966 778 raft_consensus.cc:2258] T 6bc338c4bb5940b7a0ed142751987835 P d3af45dba9b04300879578698c6a2922 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 16:20:49.683214 783 tablet_replica.cc:330] T 6bc338c4bb5940b7a0ed142751987835 P 2bf53ab985e048d19a823bea592a496d: stopping tablet replica
I20260809 16:20:49.683353 783 raft_consensus.cc:2229] T 6bc338c4bb5940b7a0ed142751987835 P 2bf53ab985e048d19a823bea592a496d [term 1 LEADER]: Raft consensus shutting down.
I20260809 16:20:49.683512 783 raft_consensus.cc:2258] T 6bc338c4bb5940b7a0ed142751987835 P 2bf53ab985e048d19a823bea592a496d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 16:20:49.683547 778 ts_tablet_manager.cc:1905] T 6bc338c4bb5940b7a0ed142751987835 P d3af45dba9b04300879578698c6a2922: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 16:20:49.684022 783 ts_tablet_manager.cc:1905] T 6bc338c4bb5940b7a0ed142751987835 P 2bf53ab985e048d19a823bea592a496d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 16:20:49.684311 21710 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrNqUCn/build/release/bin/kudu with pid 31203
I20260809 16:20:49.686249 783 ts_tablet_manager.cc:1918] T 6bc338c4bb5940b7a0ed142751987835 P 2bf53ab985e048d19a823bea592a496d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 16:20:49.686420 783 log.cc:1196] T 6bc338c4bb5940b7a0ed142751987835 P 2bf53ab985e048d19a823bea592a496d: Deleting WAL directory at /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292417722232-21710-0/minicluster-data/ts-2/wal/wals/6bc338c4bb5940b7a0ed142751987835
I20260809 16:20:49.686818 783 ts_tablet_manager.cc:1939] T 6bc338c4bb5940b7a0ed142751987835 P 2bf53ab985e048d19a823bea592a496d: Deleting consensus metadata
I20260809 16:20:49.688782 778 ts_tablet_manager.cc:1918] T 6bc338c4bb5940b7a0ed142751987835 P d3af45dba9b04300879578698c6a2922: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 16:20:49.688869 778 log.cc:1196] T 6bc338c4bb5940b7a0ed142751987835 P d3af45dba9b04300879578698c6a2922: Deleting WAL directory at /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292417722232-21710-0/minicluster-data/ts-1/wal/wals/6bc338c4bb5940b7a0ed142751987835
I20260809 16:20:49.689100 778 ts_tablet_manager.cc:1939] T 6bc338c4bb5940b7a0ed142751987835 P d3af45dba9b04300879578698c6a2922: Deleting consensus metadata
I20260809 16:20:49.689525 780 tablet_replica.cc:330] T 6bc338c4bb5940b7a0ed142751987835 P 62e7e0d86ddb442abe06e05b9bce2806: stopping tablet replica
I20260809 16:20:49.689651 780 raft_consensus.cc:2229] T 6bc338c4bb5940b7a0ed142751987835 P 62e7e0d86ddb442abe06e05b9bce2806 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 16:20:49.689790 780 raft_consensus.cc:2258] T 6bc338c4bb5940b7a0ed142751987835 P 62e7e0d86ddb442abe06e05b9bce2806 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 16:20:49.690629 780 ts_tablet_manager.cc:1905] T 6bc338c4bb5940b7a0ed142751987835 P 62e7e0d86ddb442abe06e05b9bce2806: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 16:20:49.692698 780 ts_tablet_manager.cc:1918] T 6bc338c4bb5940b7a0ed142751987835 P 62e7e0d86ddb442abe06e05b9bce2806: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 16:20:49.692776 780 log.cc:1196] T 6bc338c4bb5940b7a0ed142751987835 P 62e7e0d86ddb442abe06e05b9bce2806: Deleting WAL directory at /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292417722232-21710-0/minicluster-data/ts-0/wal/wals/6bc338c4bb5940b7a0ed142751987835
I20260809 16:20:49.693030 780 ts_tablet_manager.cc:1939] T 6bc338c4bb5940b7a0ed142751987835 P 62e7e0d86ddb442abe06e05b9bce2806: Deleting consensus metadata
W20260809 16:20:49.698084 525 connection.cc:535] server connection from 127.0.0.1:38212 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260809 16:20:49.698230 331 connection.cc:535] server connection from 127.0.0.1:43182 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260809 16:20:49.698531 32589 connection.cc:535] server connection from 127.0.0.1:44924 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260809 16:20:49.702878 709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:49.684014 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44204 (local address 127.21.51.188:40073)
0809 16:20:49.684057 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0809 16:20:49.684058 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 16:20:49.692949 (+ 8891us) server_negotiation.cc:407] Connection header received
0809 16:20:49.693057 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:20:49.693058 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:20:49.693069 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:20:49.693100 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:20:49.693939 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:49.694769 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:49.695850 (+ 1081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:49.696251 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:49.696342 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:49.698651 (+ 2309us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:20:49.698652 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:20:49.698652 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:20:49.698660 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:20:49.700612 (+ 1952us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:20:49.701027 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:20:49.701028 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:20:49.701028 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:20:49.701088 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:20:49.702024 (+ 936us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:20:49.702025 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:20:49.702026 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:20:49.702169 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:20:49.702266 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0809 16:20:49.702700 (+ 434us) server_negotiation.cc:299] Negotiation successful
0809 16:20:49.702782 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260809 16:20:49.706609 32264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:49.699650 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48436 (local address 127.21.51.189:41439)
0809 16:20:49.699693 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0809 16:20:49.699695 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:20:49.699787 (+ 92us) server_negotiation.cc:407] Connection header received
0809 16:20:49.699913 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:20:49.699914 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:20:49.699925 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:20:49.699978 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:20:49.700436 (+ 458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:49.701005 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:49.702126 (+ 1121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:49.702339 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:49.702411 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:49.703923 (+ 1512us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:20:49.703925 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:20:49.703925 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:20:49.703933 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:20:49.705440 (+ 1507us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:20:49.705776 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:20:49.705777 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:20:49.705778 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:20:49.705839 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:20:49.706039 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:20:49.706040 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:20:49.706040 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:20:49.706153 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:20:49.706256 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0809 16:20:49.706389 (+ 133us) server_negotiation.cc:299] Negotiation successful
0809 16:20:49.706473 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260809 16:20:49.714298 709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:49.706624 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44214 (local address 127.21.51.188:40073)
0809 16:20:49.706661 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0809 16:20:49.706663 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:20:49.708332 (+ 1669us) server_negotiation.cc:407] Connection header received
0809 16:20:49.708417 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:20:49.708418 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:20:49.708428 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:20:49.708473 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:20:49.709166 (+ 693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:49.709623 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:49.710322 (+ 699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:49.710519 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:49.710592 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:49.711778 (+ 1186us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:20:49.711779 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:20:49.711780 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:20:49.711786 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:20:49.713220 (+ 1434us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:20:49.713547 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:20:49.713548 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:20:49.713549 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:20:49.713606 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:20:49.713816 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:20:49.713817 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:20:49.713817 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:20:49.713921 (+ 104us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:20:49.714014 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0809 16:20:49.714121 (+ 107us) server_negotiation.cc:299] Negotiation successful
0809 16:20:49.714193 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260809 16:20:50.678447 795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:50.678142 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:40451 (local address 127.21.51.129:52105)
0809 16:20:50.678310 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0809 16:20:50.678399 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.51.190:40451: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":73,"threads_started":1}
W20260809 16:20:50.678812 32760 heartbeater.cc:644] Failed to heartbeat to 127.21.51.190:40451 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.21.51.190:40451: connect: Connection refused (error 111)
I20260809 16:20:50.679374 796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:50.679109 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:40451 (local address 127.21.51.131:40553)
0809 16:20:50.679238 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0809 16:20:50.679307 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.51.190:40451: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":86,"threads_started":1}
W20260809 16:20:50.679641 694 heartbeater.cc:644] Failed to heartbeat to 127.21.51.190:40451 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.21.51.190:40451: connect: Connection refused (error 111)
I20260809 16:20:50.680950 799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:50.680754 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:40451 (local address 127.21.51.130:44777)
0809 16:20:50.680867 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0809 16:20:50.680916 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.51.190:40451: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":64,"threads_started":1}
W20260809 16:20:50.681138 494 heartbeater.cc:644] Failed to heartbeat to 127.21.51.190:40451 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.21.51.190:40451: connect: Connection refused (error 111)
I20260809 16:20:51.182375 800 raft_consensus.cc:482] T 00000000000000000000000000000000 P e438fd6b8e6645f7addce3b7676caca0 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4896338a36d9400f84bedb07f140d7ad)
I20260809 16:20:51.182576 800 raft_consensus.cc:504] T 00000000000000000000000000000000 P e438fd6b8e6645f7addce3b7676caca0 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "4896338a36d9400f84bedb07f140d7ad" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 40451 } } peers { permanent_uuid: "e438fd6b8e6645f7addce3b7676caca0" member_type: VOTER last_known_addr { host: "127.21.51.189" port: 41439 } attrs { promote: false } } peers { permanent_uuid: "04e0582b6c844b10a2f992e33317c169" member_type: VOTER last_known_addr { host: "127.21.51.188" port: 40073 } attrs { promote: false } }
I20260809 16:20:51.182715 801 raft_consensus.cc:482] T 00000000000000000000000000000000 P 04e0582b6c844b10a2f992e33317c169 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4896338a36d9400f84bedb07f140d7ad)
I20260809 16:20:51.182792 801 raft_consensus.cc:504] T 00000000000000000000000000000000 P 04e0582b6c844b10a2f992e33317c169 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "4896338a36d9400f84bedb07f140d7ad" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 40451 } } peers { permanent_uuid: "e438fd6b8e6645f7addce3b7676caca0" member_type: VOTER last_known_addr { host: "127.21.51.189" port: 41439 } attrs { promote: false } } peers { permanent_uuid: "04e0582b6c844b10a2f992e33317c169" member_type: VOTER last_known_addr { host: "127.21.51.188" port: 40073 } attrs { promote: false } }
I20260809 16:20:51.183005 801 leader_election.cc:288] T 00000000000000000000000000000000 P 04e0582b6c844b10a2f992e33317c169 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4896338a36d9400f84bedb07f140d7ad (127.21.51.190:40451), e438fd6b8e6645f7addce3b7676caca0 (127.21.51.189:41439)
I20260809 16:20:51.183296 800 leader_election.cc:288] T 00000000000000000000000000000000 P e438fd6b8e6645f7addce3b7676caca0 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4896338a36d9400f84bedb07f140d7ad (127.21.51.190:40451), 04e0582b6c844b10a2f992e33317c169 (127.21.51.188:40073)
I20260809 16:20:51.183506 804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:51.183162 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:40451 (local address 127.0.0.1:40644)
0809 16:20:51.183280 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0809 16:20:51.183457 (+ 177us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.51.190:40451: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":63,"threads_started":1}
I20260809 16:20:51.183656 806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:51.183162 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:40451 (local address 127.0.0.1:40636)
0809 16:20:51.183450 (+ 288us) negotiation.cc:107] Waiting for socket to connect
0809 16:20:51.183606 (+ 156us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.51.190:40451: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":272,"thread_start_us":67,"threads_started":1}
W20260809 16:20:51.183722 32165 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.21.51.190:40451: connect: Connection refused (error 111)
W20260809 16:20:51.183794 32442 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.21.51.190:40451: connect: Connection refused (error 111)
I20260809 16:20:51.184234 804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:51.184106 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:40451 (local address 127.0.0.1:40656)
0809 16:20:51.184178 (+ 72us) negotiation.cc:107] Waiting for socket to connect
0809 16:20:51.184211 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.51.190:40451: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260809 16:20:51.184314 806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:51.184206 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:40451 (local address 127.0.0.1:40666)
0809 16:20:51.184258 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0809 16:20:51.184285 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.51.190:40451: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260809 16:20:51.184420 32165 leader_election.cc:334] T 00000000000000000000000000000000 P e438fd6b8e6645f7addce3b7676caca0 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 4896338a36d9400f84bedb07f140d7ad (127.21.51.190:40451): Network error: Client connection negotiation failed: client connection to 127.21.51.190:40451: connect: Connection refused (error 111)
W20260809 16:20:51.184522 32442 leader_election.cc:334] T 00000000000000000000000000000000 P 04e0582b6c844b10a2f992e33317c169 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 4896338a36d9400f84bedb07f140d7ad (127.21.51.190:40451): Network error: Client connection negotiation failed: client connection to 127.21.51.190:40451: connect: Connection refused (error 111)
I20260809 16:20:51.186753 805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:51.183337 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.189:41439 (local address 127.0.0.1:48442)
0809 16:20:51.183713 (+ 376us) negotiation.cc:107] Waiting for socket to connect
0809 16:20:51.183723 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 16:20:51.183794 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:20:51.184014 (+ 220us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:20:51.184015 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:20:51.184018 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:20:51.184079 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:20:51.184081 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:20:51.185139 (+ 1058us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:20:51.185140 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:20:51.185905 (+ 765us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:20:51.185906 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:20:51.186489 (+ 583us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:20:51.186490 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:20:51.186559 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:51.186563 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 16:20:51.186641 (+ 78us) client_negotiation.cc:240] Negotiation successful
0809 16:20:51.186688 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":361,"thread_start_us":41,"threads_started":1}
I20260809 16:20:51.186744 808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:51.183574 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48442 (local address 127.21.51.189:41439)
0809 16:20:51.183652 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0809 16:20:51.183654 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:20:51.183853 (+ 199us) server_negotiation.cc:407] Connection header received
0809 16:20:51.183883 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:20:51.183884 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:20:51.183896 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:20:51.183962 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:20:51.184162 (+ 200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:51.185044 (+ 882us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:51.186016 (+ 972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:51.186401 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:51.186489 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:51.186501 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0809 16:20:51.186645 (+ 144us) server_negotiation.cc:299] Negotiation successful
0809 16:20:51.186682 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61,"thread_start_us":37,"threads_started":1}
I20260809 16:20:51.187077 807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:51.183484 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.188:40073 (local address 127.0.0.1:44216)
0809 16:20:51.183793 (+ 309us) negotiation.cc:107] Waiting for socket to connect
0809 16:20:51.183805 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 16:20:51.183868 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:20:51.184310 (+ 442us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:20:51.184311 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:20:51.184313 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:20:51.184368 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:20:51.184370 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:20:51.185149 (+ 779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:20:51.185150 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:20:51.186131 (+ 981us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:20:51.186134 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:20:51.186782 (+ 648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:20:51.186783 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:20:51.186852 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:51.186855 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 16:20:51.186946 (+ 91us) client_negotiation.cc:240] Negotiation successful
0809 16:20:51.186990 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":295,"thread_start_us":57,"threads_started":1}
I20260809 16:20:51.187104 32201 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "04e0582b6c844b10a2f992e33317c169" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "e438fd6b8e6645f7addce3b7676caca0" is_pre_election: true
I20260809 16:20:51.187204 32201 raft_consensus.cc:2454] T 00000000000000000000000000000000 P e438fd6b8e6645f7addce3b7676caca0 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 04e0582b6c844b10a2f992e33317c169 in term 1.
I20260809 16:20:51.187249 811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:51.183661 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44216 (local address 127.21.51.188:40073)
0809 16:20:51.184165 (+ 504us) server_negotiation.cc:206] Beginning negotiation
0809 16:20:51.184167 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:20:51.184175 (+ 8us) server_negotiation.cc:407] Connection header received
0809 16:20:51.184199 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:20:51.184199 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:20:51.184209 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:20:51.184438 (+ 229us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:20:51.184474 (+ 36us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:51.185058 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:51.186302 (+ 1244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:51.186682 (+ 380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:51.186777 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:51.186787 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0809 16:20:51.187110 (+ 323us) server_negotiation.cc:299] Negotiation successful
0809 16:20:51.187164 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":487,"thread_start_us":324,"threads_started":1}
I20260809 16:20:51.187355 32442 leader_election.cc:302] T 00000000000000000000000000000000 P 04e0582b6c844b10a2f992e33317c169 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 04e0582b6c844b10a2f992e33317c169, e438fd6b8e6645f7addce3b7676caca0; no voters: 4896338a36d9400f84bedb07f140d7ad
I20260809 16:20:51.187449 801 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 04e0582b6c844b10a2f992e33317c169 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260809 16:20:51.187495 801 raft_consensus.cc:482] T 00000000000000000000000000000000 P 04e0582b6c844b10a2f992e33317c169 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 4896338a36d9400f84bedb07f140d7ad)
I20260809 16:20:51.187516 801 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 04e0582b6c844b10a2f992e33317c169 [term 1 FOLLOWER]: Advancing to term 2
I20260809 16:20:51.187474 32478 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e438fd6b8e6645f7addce3b7676caca0" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "04e0582b6c844b10a2f992e33317c169" is_pre_election: true
I20260809 16:20:51.188241 801 raft_consensus.cc:504] T 00000000000000000000000000000000 P 04e0582b6c844b10a2f992e33317c169 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "4896338a36d9400f84bedb07f140d7ad" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 40451 } } peers { permanent_uuid: "e438fd6b8e6645f7addce3b7676caca0" member_type: VOTER last_known_addr { host: "127.21.51.189" port: 41439 } attrs { promote: false } } peers { permanent_uuid: "04e0582b6c844b10a2f992e33317c169" member_type: VOTER last_known_addr { host: "127.21.51.188" port: 40073 } attrs { promote: false } }
I20260809 16:20:51.188390 32478 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 04e0582b6c844b10a2f992e33317c169 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e438fd6b8e6645f7addce3b7676caca0 in current term 2: Already voted for candidate 04e0582b6c844b10a2f992e33317c169 in this term.
I20260809 16:20:51.188403 801 leader_election.cc:288] T 00000000000000000000000000000000 P 04e0582b6c844b10a2f992e33317c169 [CANDIDATE]: Term 2 election: Requested vote from peers 4896338a36d9400f84bedb07f140d7ad (127.21.51.190:40451), e438fd6b8e6645f7addce3b7676caca0 (127.21.51.189:41439)
I20260809 16:20:51.188629 32167 leader_election.cc:302] T 00000000000000000000000000000000 P e438fd6b8e6645f7addce3b7676caca0 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e438fd6b8e6645f7addce3b7676caca0; no voters: 04e0582b6c844b10a2f992e33317c169, 4896338a36d9400f84bedb07f140d7ad
I20260809 16:20:51.188642 805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:51.188522 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:40451 (local address 127.0.0.1:40672)
0809 16:20:51.188592 (+ 70us) negotiation.cc:107] Waiting for socket to connect
0809 16:20:51.188616 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.51.190:40451: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260809 16:20:51.188833 800 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e438fd6b8e6645f7addce3b7676caca0 [term 1 FOLLOWER]: Advancing to term 2
I20260809 16:20:51.188902 32201 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "04e0582b6c844b10a2f992e33317c169" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "e438fd6b8e6645f7addce3b7676caca0"
I20260809 16:20:51.189054 805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:51.188937 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:40451 (local address 127.0.0.1:40678)
0809 16:20:51.189001 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0809 16:20:51.189029 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.51.190:40451: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":20}
W20260809 16:20:51.189131 32442 leader_election.cc:334] T 00000000000000000000000000000000 P 04e0582b6c844b10a2f992e33317c169 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 4896338a36d9400f84bedb07f140d7ad (127.21.51.190:40451): Network error: Client connection negotiation failed: client connection to 127.21.51.190:40451: connect: Connection refused (error 111)
I20260809 16:20:51.189592 800 raft_consensus.cc:2726] T 00000000000000000000000000000000 P e438fd6b8e6645f7addce3b7676caca0 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 16:20:51.190272 32201 raft_consensus.cc:2454] T 00000000000000000000000000000000 P e438fd6b8e6645f7addce3b7676caca0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 04e0582b6c844b10a2f992e33317c169 in term 2.
I20260809 16:20:51.190435 32442 leader_election.cc:302] T 00000000000000000000000000000000 P 04e0582b6c844b10a2f992e33317c169 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 04e0582b6c844b10a2f992e33317c169, e438fd6b8e6645f7addce3b7676caca0; no voters: 4896338a36d9400f84bedb07f140d7ad
I20260809 16:20:51.190546 801 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 04e0582b6c844b10a2f992e33317c169 [term 2 FOLLOWER]: Leader election won for term 2
I20260809 16:20:51.190716 801 raft_consensus.cc:686] T 00000000000000000000000000000000 P 04e0582b6c844b10a2f992e33317c169 [term 2 LEADER]: Becoming Leader. State: Replica: 04e0582b6c844b10a2f992e33317c169, State: Running, Role: LEADER
I20260809 16:20:51.190943 801 consensus_queue.cc:237] T 00000000000000000000000000000000 P 04e0582b6c844b10a2f992e33317c169 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "4896338a36d9400f84bedb07f140d7ad" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 40451 } } peers { permanent_uuid: "e438fd6b8e6645f7addce3b7676caca0" member_type: VOTER last_known_addr { host: "127.21.51.189" port: 41439 } attrs { promote: false } } peers { permanent_uuid: "04e0582b6c844b10a2f992e33317c169" member_type: VOTER last_known_addr { host: "127.21.51.188" port: 40073 } attrs { promote: false } }
I20260809 16:20:51.191416 816 sys_catalog.cc:455] T 00000000000000000000000000000000 P 04e0582b6c844b10a2f992e33317c169 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 04e0582b6c844b10a2f992e33317c169. Latest consensus state: current_term: 2 leader_uuid: "04e0582b6c844b10a2f992e33317c169" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "4896338a36d9400f84bedb07f140d7ad" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 40451 } } peers { permanent_uuid: "e438fd6b8e6645f7addce3b7676caca0" member_type: VOTER last_known_addr { host: "127.21.51.189" port: 41439 } attrs { promote: false } } peers { permanent_uuid: "04e0582b6c844b10a2f992e33317c169" member_type: VOTER last_known_addr { host: "127.21.51.188" port: 40073 } attrs { promote: false } } }
I20260809 16:20:51.191520 816 sys_catalog.cc:458] T 00000000000000000000000000000000 P 04e0582b6c844b10a2f992e33317c169 [sys.catalog]: This master's current role is: LEADER
I20260809 16:20:51.192041 819 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 16:20:51.192533 819 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 16:20:51.192600 819 catalog_manager.cc:1259] Loaded cluster ID: 70b9d0b7af95488b90d1785aca0f4ff6
I20260809 16:20:51.192617 819 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 16:20:51.192837 819 catalog_manager.cc:1504] Loading token signing keys...
I20260809 16:20:51.192909 819 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 04e0582b6c844b10a2f992e33317c169: Loaded TSK: 0
I20260809 16:20:51.193008 819 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 16:20:51.255617 32465 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44214:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 16:20:51.257391 32465 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 16:20:51.258468 32201 raft_consensus.cc:1261] T 00000000000000000000000000000000 P e438fd6b8e6645f7addce3b7676caca0 [term 2 FOLLOWER]: Refusing update from remote peer 04e0582b6c844b10a2f992e33317c169: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 2 index: 14. (index mismatch)
I20260809 16:20:51.258558 805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:51.258413 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:40451 (local address 127.0.0.1:40696)
0809 16:20:51.258472 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0809 16:20:51.258507 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.51.190:40451: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260809 16:20:51.258736 816 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 04e0582b6c844b10a2f992e33317c169 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e438fd6b8e6645f7addce3b7676caca0" member_type: VOTER last_known_addr { host: "127.21.51.189" port: 41439 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260809 16:20:51.258916 805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:51.258815 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:40451 (local address 127.0.0.1:40700)
0809 16:20:51.258867 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0809 16:20:51.258894 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.51.190:40451: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260809 16:20:51.259018 32442 consensus_peers.cc:487] T 00000000000000000000000000000000 P 04e0582b6c844b10a2f992e33317c169 -> Peer 4896338a36d9400f84bedb07f140d7ad (127.21.51.190:40451): Couldn't send request to peer 4896338a36d9400f84bedb07f140d7ad. Status: Network error: Client connection negotiation failed: client connection to 127.21.51.190:40451: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260809 16:20:51.259423 800 sys_catalog.cc:455] T 00000000000000000000000000000000 P e438fd6b8e6645f7addce3b7676caca0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 04e0582b6c844b10a2f992e33317c169. Latest consensus state: current_term: 2 leader_uuid: "04e0582b6c844b10a2f992e33317c169" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "4896338a36d9400f84bedb07f140d7ad" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 40451 } } peers { permanent_uuid: "e438fd6b8e6645f7addce3b7676caca0" member_type: VOTER last_known_addr { host: "127.21.51.189" port: 41439 } attrs { promote: false } } peers { permanent_uuid: "04e0582b6c844b10a2f992e33317c169" member_type: VOTER last_known_addr { host: "127.21.51.188" port: 40073 } attrs { promote: false } } }
I20260809 16:20:51.259526 800 sys_catalog.cc:458] T 00000000000000000000000000000000 P e438fd6b8e6645f7addce3b7676caca0 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 16:20:51.259917 820 mvcc.cc:205] Tried to move back new op lower bound from 7316653880352526336 to 7316653880078876672. Current Snapshot: MvccSnapshot[applied={T|T < 7316653880352526336}]
I20260809 16:20:51.260010 801 sys_catalog.cc:455] T 00000000000000000000000000000000 P 04e0582b6c844b10a2f992e33317c169 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "04e0582b6c844b10a2f992e33317c169" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "4896338a36d9400f84bedb07f140d7ad" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 40451 } } peers { permanent_uuid: "e438fd6b8e6645f7addce3b7676caca0" member_type: VOTER last_known_addr { host: "127.21.51.189" port: 41439 } attrs { promote: false } } peers { permanent_uuid: "04e0582b6c844b10a2f992e33317c169" member_type: VOTER last_known_addr { host: "127.21.51.188" port: 40073 } attrs { promote: false } } }
I20260809 16:20:51.260100 801 sys_catalog.cc:458] T 00000000000000000000000000000000 P 04e0582b6c844b10a2f992e33317c169 [sys.catalog]: This master's current role is: LEADER
I20260809 16:20:51.260217 800 sys_catalog.cc:455] T 00000000000000000000000000000000 P e438fd6b8e6645f7addce3b7676caca0 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "04e0582b6c844b10a2f992e33317c169" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "4896338a36d9400f84bedb07f140d7ad" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 40451 } } peers { permanent_uuid: "e438fd6b8e6645f7addce3b7676caca0" member_type: VOTER last_known_addr { host: "127.21.51.189" port: 41439 } attrs { promote: false } } peers { permanent_uuid: "04e0582b6c844b10a2f992e33317c169" member_type: VOTER last_known_addr { host: "127.21.51.188" port: 40073 } attrs { promote: false } } }
I20260809 16:20:51.260317 800 sys_catalog.cc:458] T 00000000000000000000000000000000 P e438fd6b8e6645f7addce3b7676caca0 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 16:20:51.260594 821 mvcc.cc:205] Tried to move back new op lower bound from 7316653880352526336 to 7316653880078876672. Current Snapshot: MvccSnapshot[applied={T|T < 7316653880352526336 or (T in {7316653880352526336})}]
I20260809 16:20:51.265407 829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:51.261914 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38232 (local address 127.21.51.131:46809)
0809 16:20:51.262652 (+ 738us) server_negotiation.cc:206] Beginning negotiation
0809 16:20:51.262654 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:20:51.262663 (+ 9us) server_negotiation.cc:407] Connection header received
0809 16:20:51.262705 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:20:51.262705 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:20:51.262716 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:20:51.262793 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:20:51.262927 (+ 134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:51.263619 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:51.264508 (+ 889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:51.265052 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:51.265139 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:51.265153 (+ 14us) server_negotiation.cc:1035] Waiting for connection context
0809 16:20:51.265300 (+ 147us) server_negotiation.cc:299] Negotiation successful
0809 16:20:51.265343 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":714,"thread_start_us":71,"threads_started":1}
I20260809 16:20:51.265398 805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:51.261855 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.131:46809 (local address 127.0.0.1:38232)
0809 16:20:51.261891 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0809 16:20:51.261903 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 16:20:51.261944 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:20:51.262802 (+ 858us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:20:51.262803 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:20:51.262805 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:20:51.262843 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:20:51.262844 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:20:51.263713 (+ 869us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:20:51.263714 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:20:51.264415 (+ 701us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:20:51.264417 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:20:51.265137 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:20:51.265138 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:20:51.265200 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:51.265205 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 16:20:51.265291 (+ 86us) client_negotiation.cc:240] Negotiation successful
0809 16:20:51.265334 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":20}
I20260809 16:20:51.265497 806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:51.261896 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.130:34781 (local address 127.0.0.1:43192)
0809 16:20:51.261992 (+ 96us) negotiation.cc:107] Waiting for socket to connect
0809 16:20:51.262002 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 16:20:51.262039 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:20:51.262383 (+ 344us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:20:51.262384 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:20:51.262387 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:20:51.262441 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:20:51.262443 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:20:51.263273 (+ 830us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:20:51.263274 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:20:51.264419 (+ 1145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:20:51.264422 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:20:51.265271 (+ 849us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:20:51.265272 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:20:51.265342 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:51.265346 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 16:20:51.265377 (+ 31us) client_negotiation.cc:240] Negotiation successful
0809 16:20:51.265419 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":81}
I20260809 16:20:51.265614 832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:51.261970 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.129:33587 (local address 127.0.0.1:44940)
0809 16:20:51.262107 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0809 16:20:51.262117 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 16:20:51.262162 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:20:51.262565 (+ 403us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:20:51.262566 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:20:51.262567 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:20:51.262597 (+ 30us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:20:51.262607 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:20:51.263535 (+ 928us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:20:51.263536 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:20:51.264609 (+ 1073us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:20:51.264611 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:20:51.265399 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:20:51.265400 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:20:51.265451 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:51.265454 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 16:20:51.265499 (+ 45us) client_negotiation.cc:240] Negotiation successful
0809 16:20:51.265539 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"mutex_wait_us":28,"thread_start_us":79,"threads_started":1}
I20260809 16:20:51.265630 599 tablet_service.cc:1467] Processing CreateTablet for tablet 73ed14def558470da58db9a09aa4d3ad (DEFAULT_TABLE table=test-table [id=d3aa9f2a6dd340eabc6dd79b7db5a10c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:20:51.265761 599 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 73ed14def558470da58db9a09aa4d3ad. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:20:51.265842 834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:51.262161 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44940 (local address 127.21.51.129:33587)
0809 16:20:51.262455 (+ 294us) server_negotiation.cc:206] Beginning negotiation
0809 16:20:51.262457 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:20:51.262464 (+ 7us) server_negotiation.cc:407] Connection header received
0809 16:20:51.262489 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:20:51.262490 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:20:51.262503 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:20:51.262572 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:20:51.262700 (+ 128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:51.263451 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:51.264726 (+ 1275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:51.265285 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:51.265343 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:51.265356 (+ 13us) server_negotiation.cc:1035] Waiting for connection context
0809 16:20:51.265697 (+ 341us) server_negotiation.cc:299] Negotiation successful
0809 16:20:51.265755 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":271,"thread_start_us":57,"threads_started":1}
I20260809 16:20:51.266065 833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:51.262136 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43192 (local address 127.21.51.130:34781)
0809 16:20:51.262263 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0809 16:20:51.262264 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 16:20:51.262272 (+ 8us) server_negotiation.cc:407] Connection header received
0809 16:20:51.262301 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:20:51.262302 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:20:51.262315 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:20:51.262371 (+ 56us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:20:51.262504 (+ 133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:51.263193 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:51.264557 (+ 1364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:51.265171 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:51.265837 (+ 666us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:51.265852 (+ 15us) server_negotiation.cc:1035] Waiting for connection context
0809 16:20:51.265872 (+ 20us) server_negotiation.cc:299] Negotiation successful
0809 16:20:51.266004 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":111,"thread_start_us":77,"threads_started":1}
I20260809 16:20:51.266321 398 tablet_service.cc:1467] Processing CreateTablet for tablet 73ed14def558470da58db9a09aa4d3ad (DEFAULT_TABLE table=test-table [id=d3aa9f2a6dd340eabc6dd79b7db5a10c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:20:51.266470 398 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 73ed14def558470da58db9a09aa4d3ad. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:20:51.267242 836 tablet_bootstrap.cc:492] T 73ed14def558470da58db9a09aa4d3ad P 2bf53ab985e048d19a823bea592a496d: Bootstrap starting.
I20260809 16:20:51.267491 32664 tablet_service.cc:1467] Processing CreateTablet for tablet 73ed14def558470da58db9a09aa4d3ad (DEFAULT_TABLE table=test-table [id=d3aa9f2a6dd340eabc6dd79b7db5a10c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:20:51.267621 32664 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 73ed14def558470da58db9a09aa4d3ad. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:20:51.267647 836 tablet_bootstrap.cc:654] T 73ed14def558470da58db9a09aa4d3ad P 2bf53ab985e048d19a823bea592a496d: Neither blocks nor log segments found. Creating new log.
I20260809 16:20:51.267830 838 tablet_bootstrap.cc:492] T 73ed14def558470da58db9a09aa4d3ad P d3af45dba9b04300879578698c6a2922: Bootstrap starting.
I20260809 16:20:51.268121 838 tablet_bootstrap.cc:654] T 73ed14def558470da58db9a09aa4d3ad P d3af45dba9b04300879578698c6a2922: Neither blocks nor log segments found. Creating new log.
I20260809 16:20:51.268674 836 tablet_bootstrap.cc:492] T 73ed14def558470da58db9a09aa4d3ad P 2bf53ab985e048d19a823bea592a496d: No bootstrap required, opened a new log
I20260809 16:20:51.268764 836 ts_tablet_manager.cc:1397] T 73ed14def558470da58db9a09aa4d3ad P 2bf53ab985e048d19a823bea592a496d: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 16:20:51.268766 838 tablet_bootstrap.cc:492] T 73ed14def558470da58db9a09aa4d3ad P d3af45dba9b04300879578698c6a2922: No bootstrap required, opened a new log
I20260809 16:20:51.268816 838 ts_tablet_manager.cc:1397] T 73ed14def558470da58db9a09aa4d3ad P d3af45dba9b04300879578698c6a2922: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260809 16:20:51.268899 841 tablet_bootstrap.cc:492] T 73ed14def558470da58db9a09aa4d3ad P 62e7e0d86ddb442abe06e05b9bce2806: Bootstrap starting.
I20260809 16:20:51.268977 838 raft_consensus.cc:348] T 73ed14def558470da58db9a09aa4d3ad P d3af45dba9b04300879578698c6a2922 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2bf53ab985e048d19a823bea592a496d" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 46809 } } peers { permanent_uuid: "d3af45dba9b04300879578698c6a2922" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 34781 } } peers { permanent_uuid: "62e7e0d86ddb442abe06e05b9bce2806" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 33587 } }
I20260809 16:20:51.269054 838 raft_consensus.cc:374] T 73ed14def558470da58db9a09aa4d3ad P d3af45dba9b04300879578698c6a2922 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:20:51.269014 836 raft_consensus.cc:348] T 73ed14def558470da58db9a09aa4d3ad P 2bf53ab985e048d19a823bea592a496d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2bf53ab985e048d19a823bea592a496d" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 46809 } } peers { permanent_uuid: "d3af45dba9b04300879578698c6a2922" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 34781 } } peers { permanent_uuid: "62e7e0d86ddb442abe06e05b9bce2806" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 33587 } }
I20260809 16:20:51.269084 838 raft_consensus.cc:729] T 73ed14def558470da58db9a09aa4d3ad P d3af45dba9b04300879578698c6a2922 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d3af45dba9b04300879578698c6a2922, State: Initialized, Role: FOLLOWER
I20260809 16:20:51.269094 836 raft_consensus.cc:374] T 73ed14def558470da58db9a09aa4d3ad P 2bf53ab985e048d19a823bea592a496d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:20:51.269114 836 raft_consensus.cc:729] T 73ed14def558470da58db9a09aa4d3ad P 2bf53ab985e048d19a823bea592a496d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2bf53ab985e048d19a823bea592a496d, State: Initialized, Role: FOLLOWER
I20260809 16:20:51.269127 838 consensus_queue.cc:260] T 73ed14def558470da58db9a09aa4d3ad P d3af45dba9b04300879578698c6a2922 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2bf53ab985e048d19a823bea592a496d" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 46809 } } peers { permanent_uuid: "d3af45dba9b04300879578698c6a2922" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 34781 } } peers { permanent_uuid: "62e7e0d86ddb442abe06e05b9bce2806" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 33587 } }
I20260809 16:20:51.269181 836 consensus_queue.cc:260] T 73ed14def558470da58db9a09aa4d3ad P 2bf53ab985e048d19a823bea592a496d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2bf53ab985e048d19a823bea592a496d" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 46809 } } peers { permanent_uuid: "d3af45dba9b04300879578698c6a2922" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 34781 } } peers { permanent_uuid: "62e7e0d86ddb442abe06e05b9bce2806" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 33587 } }
I20260809 16:20:51.269266 841 tablet_bootstrap.cc:654] T 73ed14def558470da58db9a09aa4d3ad P 62e7e0d86ddb442abe06e05b9bce2806: Neither blocks nor log segments found. Creating new log.
I20260809 16:20:51.269289 838 ts_tablet_manager.cc:1428] T 73ed14def558470da58db9a09aa4d3ad P d3af45dba9b04300879578698c6a2922: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260809 16:20:51.269366 836 ts_tablet_manager.cc:1428] T 73ed14def558470da58db9a09aa4d3ad P 2bf53ab985e048d19a823bea592a496d: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260809 16:20:51.269799 841 tablet_bootstrap.cc:492] T 73ed14def558470da58db9a09aa4d3ad P 62e7e0d86ddb442abe06e05b9bce2806: No bootstrap required, opened a new log
I20260809 16:20:51.269856 841 ts_tablet_manager.cc:1397] T 73ed14def558470da58db9a09aa4d3ad P 62e7e0d86ddb442abe06e05b9bce2806: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260809 16:20:51.269979 841 raft_consensus.cc:348] T 73ed14def558470da58db9a09aa4d3ad P 62e7e0d86ddb442abe06e05b9bce2806 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2bf53ab985e048d19a823bea592a496d" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 46809 } } peers { permanent_uuid: "d3af45dba9b04300879578698c6a2922" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 34781 } } peers { permanent_uuid: "62e7e0d86ddb442abe06e05b9bce2806" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 33587 } }
I20260809 16:20:51.270057 841 raft_consensus.cc:374] T 73ed14def558470da58db9a09aa4d3ad P 62e7e0d86ddb442abe06e05b9bce2806 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:20:51.270090 841 raft_consensus.cc:729] T 73ed14def558470da58db9a09aa4d3ad P 62e7e0d86ddb442abe06e05b9bce2806 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 62e7e0d86ddb442abe06e05b9bce2806, State: Initialized, Role: FOLLOWER
I20260809 16:20:51.270133 841 consensus_queue.cc:260] T 73ed14def558470da58db9a09aa4d3ad P 62e7e0d86ddb442abe06e05b9bce2806 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2bf53ab985e048d19a823bea592a496d" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 46809 } } peers { permanent_uuid: "d3af45dba9b04300879578698c6a2922" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 34781 } } peers { permanent_uuid: "62e7e0d86ddb442abe06e05b9bce2806" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 33587 } }
I20260809 16:20:51.270295 841 ts_tablet_manager.cc:1428] T 73ed14def558470da58db9a09aa4d3ad P 62e7e0d86ddb442abe06e05b9bce2806: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260809 16:20:51.270634 850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:51.269843 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:40451 (local address 127.21.51.131:36465)
0809 16:20:51.270564 (+ 721us) negotiation.cc:107] Waiting for socket to connect
0809 16:20:51.270599 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.51.190:40451: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":701,"thread_start_us":83,"threads_started":1}
I20260809 16:20:51.270888 691 heartbeater.cc:503] Master 127.21.51.188:40073 was elected leader, sending a full tablet report...
I20260809 16:20:51.271158 849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:51.269559 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:40451 (local address 127.21.51.130:59157)
0809 16:20:51.270917 (+ 1358us) negotiation.cc:107] Waiting for socket to connect
0809 16:20:51.271109 (+ 192us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.51.190:40451: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1335,"thread_start_us":80,"threads_started":1}
I20260809 16:20:51.271230 850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:51.271144 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:40451 (local address 127.21.51.131:57737)
0809 16:20:51.271176 (+ 32us) negotiation.cc:107] Waiting for socket to connect
0809 16:20:51.271203 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.51.190:40451: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":17}
I20260809 16:20:51.271430 852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:51.270603 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:40451 (local address 127.21.51.129:46123)
0809 16:20:51.271357 (+ 754us) negotiation.cc:107] Waiting for socket to connect
0809 16:20:51.271391 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.51.190:40451: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":727,"thread_start_us":81,"threads_started":1}
I20260809 16:20:51.512885 843 raft_consensus.cc:482] T 73ed14def558470da58db9a09aa4d3ad P d3af45dba9b04300879578698c6a2922 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 16:20:51.513005 843 raft_consensus.cc:504] T 73ed14def558470da58db9a09aa4d3ad P d3af45dba9b04300879578698c6a2922 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2bf53ab985e048d19a823bea592a496d" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 46809 } } peers { permanent_uuid: "d3af45dba9b04300879578698c6a2922" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 34781 } } peers { permanent_uuid: "62e7e0d86ddb442abe06e05b9bce2806" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 33587 } }
I20260809 16:20:51.513368 843 leader_election.cc:288] T 73ed14def558470da58db9a09aa4d3ad P d3af45dba9b04300879578698c6a2922 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2bf53ab985e048d19a823bea592a496d (127.21.51.131:46809), 62e7e0d86ddb442abe06e05b9bce2806 (127.21.51.129:33587)
I20260809 16:20:51.517637 829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:51.513789 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.51.130:52915 (local address 127.21.51.131:46809)
0809 16:20:51.513840 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0809 16:20:51.513842 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:20:51.514048 (+ 206us) server_negotiation.cc:407] Connection header received
0809 16:20:51.514107 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:20:51.514109 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:20:51.514121 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:20:51.514176 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:20:51.514317 (+ 141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:51.515109 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:51.516631 (+ 1522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:51.517252 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:51.517331 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:51.517342 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0809 16:20:51.517518 (+ 176us) server_negotiation.cc:299] Negotiation successful
0809 16:20:51.517563 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260809 16:20:51.517637 834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:51.513782 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.51.130:53269 (local address 127.21.51.129:33587)
0809 16:20:51.513845 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0809 16:20:51.513847 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:20:51.513884 (+ 37us) server_negotiation.cc:407] Connection header received
0809 16:20:51.513988 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:20:51.513989 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:20:51.514005 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:20:51.514083 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:20:51.514318 (+ 235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:51.515077 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:51.516631 (+ 1554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:51.517243 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:51.517332 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:51.517342 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0809 16:20:51.517517 (+ 175us) server_negotiation.cc:299] Negotiation successful
0809 16:20:51.517562 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260809 16:20:51.517652 849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:51.513677 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.129:33587 (local address 127.21.51.130:53269)
0809 16:20:51.513818 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0809 16:20:51.513833 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 16:20:51.513903 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:20:51.514107 (+ 204us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:20:51.514108 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:20:51.514110 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:20:51.514217 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:20:51.514220 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:20:51.515176 (+ 956us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:20:51.515177 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:20:51.516513 (+ 1336us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:20:51.516516 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:20:51.517337 (+ 821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:20:51.517338 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:20:51.517427 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:51.517432 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 16:20:51.517496 (+ 64us) client_negotiation.cc:240] Negotiation successful
0809 16:20:51.517552 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"mutex_wait_us":36}
I20260809 16:20:51.517669 854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:51.513677 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.131:46809 (local address 127.21.51.130:52915)
0809 16:20:51.513951 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0809 16:20:51.513961 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 16:20:51.514035 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:20:51.514189 (+ 154us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:20:51.514190 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:20:51.514192 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:20:51.514247 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:20:51.514248 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:20:51.515179 (+ 931us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:20:51.515180 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:20:51.516513 (+ 1333us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:20:51.516516 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:20:51.517359 (+ 843us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:20:51.517360 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:20:51.517427 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:51.517433 (+ 6us) client_negotiation.cc:769] Sending connection context
0809 16:20:51.517496 (+ 63us) client_negotiation.cc:240] Negotiation successful
0809 16:20:51.517552 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":252,"mutex_wait_us":164,"thread_start_us":117,"threads_started":1}
I20260809 16:20:51.517997 32690 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "73ed14def558470da58db9a09aa4d3ad" candidate_uuid: "d3af45dba9b04300879578698c6a2922" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "62e7e0d86ddb442abe06e05b9bce2806" is_pre_election: true
I20260809 16:20:51.518107 32690 raft_consensus.cc:2454] T 73ed14def558470da58db9a09aa4d3ad P 62e7e0d86ddb442abe06e05b9bce2806 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3af45dba9b04300879578698c6a2922 in term 0.
I20260809 16:20:51.518214 630 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "73ed14def558470da58db9a09aa4d3ad" candidate_uuid: "d3af45dba9b04300879578698c6a2922" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2bf53ab985e048d19a823bea592a496d" is_pre_election: true
I20260809 16:20:51.518356 334 leader_election.cc:302] T 73ed14def558470da58db9a09aa4d3ad P d3af45dba9b04300879578698c6a2922 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 62e7e0d86ddb442abe06e05b9bce2806, d3af45dba9b04300879578698c6a2922; no voters:
I20260809 16:20:51.518389 630 raft_consensus.cc:2454] T 73ed14def558470da58db9a09aa4d3ad P 2bf53ab985e048d19a823bea592a496d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3af45dba9b04300879578698c6a2922 in term 0.
I20260809 16:20:51.518551 843 raft_consensus.cc:2781] T 73ed14def558470da58db9a09aa4d3ad P d3af45dba9b04300879578698c6a2922 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 16:20:51.518608 843 raft_consensus.cc:482] T 73ed14def558470da58db9a09aa4d3ad P d3af45dba9b04300879578698c6a2922 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 16:20:51.518635 843 raft_consensus.cc:3037] T 73ed14def558470da58db9a09aa4d3ad P d3af45dba9b04300879578698c6a2922 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:20:51.519362 843 raft_consensus.cc:504] T 73ed14def558470da58db9a09aa4d3ad P d3af45dba9b04300879578698c6a2922 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2bf53ab985e048d19a823bea592a496d" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 46809 } } peers { permanent_uuid: "d3af45dba9b04300879578698c6a2922" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 34781 } } peers { permanent_uuid: "62e7e0d86ddb442abe06e05b9bce2806" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 33587 } }
I20260809 16:20:51.519521 843 leader_election.cc:288] T 73ed14def558470da58db9a09aa4d3ad P d3af45dba9b04300879578698c6a2922 [CANDIDATE]: Term 1 election: Requested vote from peers 2bf53ab985e048d19a823bea592a496d (127.21.51.131:46809), 62e7e0d86ddb442abe06e05b9bce2806 (127.21.51.129:33587)
I20260809 16:20:51.519670 630 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "73ed14def558470da58db9a09aa4d3ad" candidate_uuid: "d3af45dba9b04300879578698c6a2922" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2bf53ab985e048d19a823bea592a496d"
I20260809 16:20:51.519757 630 raft_consensus.cc:3037] T 73ed14def558470da58db9a09aa4d3ad P 2bf53ab985e048d19a823bea592a496d [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:20:51.519737 32690 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "73ed14def558470da58db9a09aa4d3ad" candidate_uuid: "d3af45dba9b04300879578698c6a2922" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "62e7e0d86ddb442abe06e05b9bce2806"
I20260809 16:20:51.519840 32690 raft_consensus.cc:3037] T 73ed14def558470da58db9a09aa4d3ad P 62e7e0d86ddb442abe06e05b9bce2806 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:20:51.520427 32690 raft_consensus.cc:2454] T 73ed14def558470da58db9a09aa4d3ad P 62e7e0d86ddb442abe06e05b9bce2806 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3af45dba9b04300879578698c6a2922 in term 1.
I20260809 16:20:51.520593 630 raft_consensus.cc:2454] T 73ed14def558470da58db9a09aa4d3ad P 2bf53ab985e048d19a823bea592a496d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3af45dba9b04300879578698c6a2922 in term 1.
I20260809 16:20:51.520670 334 leader_election.cc:302] T 73ed14def558470da58db9a09aa4d3ad P d3af45dba9b04300879578698c6a2922 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 62e7e0d86ddb442abe06e05b9bce2806, d3af45dba9b04300879578698c6a2922; no voters:
I20260809 16:20:51.520804 843 raft_consensus.cc:2781] T 73ed14def558470da58db9a09aa4d3ad P d3af45dba9b04300879578698c6a2922 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:20:51.521029 843 raft_consensus.cc:686] T 73ed14def558470da58db9a09aa4d3ad P d3af45dba9b04300879578698c6a2922 [term 1 LEADER]: Becoming Leader. State: Replica: d3af45dba9b04300879578698c6a2922, State: Running, Role: LEADER
I20260809 16:20:51.521136 492 heartbeater.cc:503] Master 127.21.51.188:40073 was elected leader, sending a full tablet report...
I20260809 16:20:51.521253 843 consensus_queue.cc:237] T 73ed14def558470da58db9a09aa4d3ad P d3af45dba9b04300879578698c6a2922 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2bf53ab985e048d19a823bea592a496d" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 46809 } } peers { permanent_uuid: "d3af45dba9b04300879578698c6a2922" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 34781 } } peers { permanent_uuid: "62e7e0d86ddb442abe06e05b9bce2806" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 33587 } }
I20260809 16:20:51.521381 854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:51.521252 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:40451 (local address 127.21.51.130:32805)
0809 16:20:51.521285 (+ 33us) negotiation.cc:107] Waiting for socket to connect
0809 16:20:51.521327 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.51.190:40451: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":17}
I20260809 16:20:51.522068 32463 catalog_manager.cc:5495] T 73ed14def558470da58db9a09aa4d3ad P d3af45dba9b04300879578698c6a2922 reported cstate change: term changed from 0 to 1, leader changed from <none> to d3af45dba9b04300879578698c6a2922 (127.21.51.130). New cstate: current_term: 1 leader_uuid: "d3af45dba9b04300879578698c6a2922" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2bf53ab985e048d19a823bea592a496d" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 46809 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d3af45dba9b04300879578698c6a2922" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 34781 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "62e7e0d86ddb442abe06e05b9bce2806" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 33587 } health_report { overall_health: UNKNOWN } } }
I20260809 16:20:51.549013 833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:51.545981 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43204 (local address 127.21.51.130:34781)
0809 16:20:51.546093 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0809 16:20:51.546095 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:20:51.546103 (+ 8us) server_negotiation.cc:407] Connection header received
0809 16:20:51.546146 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:20:51.546146 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:20:51.546165 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:20:51.546235 (+ 70us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 16:20:51.546412 (+ 177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:51.547046 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:51.548015 (+ 969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:51.548295 (+ 280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:51.548359 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:51.548542 (+ 183us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 16:20:51.548658 (+ 116us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 16:20:51.548837 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0809 16:20:51.548886 (+ 49us) server_negotiation.cc:299] Negotiation successful
0809 16:20:51.548913 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260809 16:20:51.550266 32690 raft_consensus.cc:1261] T 73ed14def558470da58db9a09aa4d3ad P 62e7e0d86ddb442abe06e05b9bce2806 [term 1 FOLLOWER]: Refusing update from remote peer d3af45dba9b04300879578698c6a2922: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 16:20:51.550357 630 raft_consensus.cc:1261] T 73ed14def558470da58db9a09aa4d3ad P 2bf53ab985e048d19a823bea592a496d [term 1 FOLLOWER]: Refusing update from remote peer d3af45dba9b04300879578698c6a2922: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 16:20:51.550581 843 consensus_queue.cc:1035] T 73ed14def558470da58db9a09aa4d3ad P d3af45dba9b04300879578698c6a2922 [LEADER]: Connected to new peer: Peer: permanent_uuid: "62e7e0d86ddb442abe06e05b9bce2806" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 33587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 16:20:51.550714 856 consensus_queue.cc:1035] T 73ed14def558470da58db9a09aa4d3ad P d3af45dba9b04300879578698c6a2922 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2bf53ab985e048d19a823bea592a496d" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 46809 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 16:20:51.551205 32759 heartbeater.cc:503] Master 127.21.51.188:40073 was elected leader, sending a full tablet report...
I20260809 16:20:51.551496 852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:51.551343 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:40451 (local address 127.21.51.129:43613)
0809 16:20:51.551397 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0809 16:20:51.551432 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.51.190:40451: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260809 16:20:51.551751 850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:51.551616 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:40451 (local address 127.21.51.131:52807)
0809 16:20:51.551675 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0809 16:20:51.551713 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.51.190:40451: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260809 16:20:51.553426 850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:51.553306 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:40451 (local address 127.21.51.131:39617)
0809 16:20:51.553370 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0809 16:20:51.553394 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.51.190:40451: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260809 16:20:51.554129 854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:51.554018 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:40451 (local address 127.21.51.130:48777)
0809 16:20:51.554053 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0809 16:20:51.554096 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.51.190:40451: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":19}
I20260809 16:20:51.554165 852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:51.554035 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:40451 (local address 127.21.51.129:50659)
0809 16:20:51.554105 (+ 70us) negotiation.cc:107] Waiting for socket to connect
0809 16:20:51.554134 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.51.190:40451: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260809 16:20:51.555552 854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:51.555462 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:40451 (local address 127.21.51.130:37035)
0809 16:20:51.555500 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0809 16:20:51.555525 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.51.190:40451: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
I20260809 16:20:51.555634 32465 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44214:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 16:20:51.555730 32465 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44214:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 16:20:51.555999 868 mvcc.cc:205] Tried to move back new op lower bound from 7316653881547456512 to 7316653881431584768. Current Snapshot: MvccSnapshot[applied={T|T < 7316653881547456512}]
I20260809 16:20:51.557056 32465 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 04e0582b6c844b10a2f992e33317c169: Sending DeleteTablet for 3 replicas of tablet 73ed14def558470da58db9a09aa4d3ad
I20260809 16:20:51.557364 599 tablet_service.cc:1514] Processing DeleteTablet for tablet 73ed14def558470da58db9a09aa4d3ad with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:20:51 UTC) from {username='oryx', principal='oryx/127.21.51.188@KRBTEST.COM'} at 127.0.0.1:38232
I20260809 16:20:51.557554 398 tablet_service.cc:1514] Processing DeleteTablet for tablet 73ed14def558470da58db9a09aa4d3ad with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:20:51 UTC) from {username='oryx', principal='oryx/127.21.51.188@KRBTEST.COM'} at 127.0.0.1:43192
I20260809 16:20:51.557575 32664 tablet_service.cc:1514] Processing DeleteTablet for tablet 73ed14def558470da58db9a09aa4d3ad with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:20:51 UTC) from {username='oryx', principal='oryx/127.21.51.188@KRBTEST.COM'} at 127.0.0.1:44940
I20260809 16:20:51.557739 886 tablet_replica.cc:330] T 73ed14def558470da58db9a09aa4d3ad P d3af45dba9b04300879578698c6a2922: stopping tablet replica
I20260809 16:20:51.557843 886 raft_consensus.cc:2229] T 73ed14def558470da58db9a09aa4d3ad P d3af45dba9b04300879578698c6a2922 [term 1 LEADER]: Raft consensus shutting down.
I20260809 16:20:51.557930 886 raft_consensus.cc:2258] T 73ed14def558470da58db9a09aa4d3ad P d3af45dba9b04300879578698c6a2922 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 16:20:51.558068 21710 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrNqUCn/build/release/bin/kudu with pid 32417
I20260809 16:20:51.558216 886 ts_tablet_manager.cc:1905] T 73ed14def558470da58db9a09aa4d3ad P d3af45dba9b04300879578698c6a2922: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 16:20:51.560680 886 ts_tablet_manager.cc:1918] T 73ed14def558470da58db9a09aa4d3ad P d3af45dba9b04300879578698c6a2922: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 16:20:51.560751 886 log.cc:1196] T 73ed14def558470da58db9a09aa4d3ad P d3af45dba9b04300879578698c6a2922: Deleting WAL directory at /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292417722232-21710-0/minicluster-data/ts-1/wal/wals/73ed14def558470da58db9a09aa4d3ad
I20260809 16:20:51.560959 886 ts_tablet_manager.cc:1939] T 73ed14def558470da58db9a09aa4d3ad P d3af45dba9b04300879578698c6a2922: Deleting consensus metadata
I20260809 16:20:51.561221 32442 catalog_manager.cc:4841] TS d3af45dba9b04300879578698c6a2922 (127.21.51.130:34781): tablet 73ed14def558470da58db9a09aa4d3ad (table test-table [id=d3aa9f2a6dd340eabc6dd79b7db5a10c]) successfully deleted
I20260809 16:20:51.566689 885 tablet_replica.cc:330] T 73ed14def558470da58db9a09aa4d3ad P 2bf53ab985e048d19a823bea592a496d: stopping tablet replica
I20260809 16:20:51.566781 885 raft_consensus.cc:2229] T 73ed14def558470da58db9a09aa4d3ad P 2bf53ab985e048d19a823bea592a496d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 16:20:51.566841 885 raft_consensus.cc:2258] T 73ed14def558470da58db9a09aa4d3ad P 2bf53ab985e048d19a823bea592a496d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 16:20:51.571256 832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:51.571143 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.129:33587 (local address 127.0.0.1:44950)
0809 16:20:51.571192 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0809 16:20:51.571220 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.51.129:33587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
I20260809 16:20:51.571257 21710 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrNqUCn/build/release/bin/kudu with pid 305
I20260809 16:20:51.572736 885 ts_tablet_manager.cc:1905] T 73ed14def558470da58db9a09aa4d3ad P 2bf53ab985e048d19a823bea592a496d: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260809 16:20:51.576747 32445 catalog_manager.cc:4584] TS 62e7e0d86ddb442abe06e05b9bce2806 (127.21.51.129:33587): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 73ed14def558470da58db9a09aa4d3ad: Network error: Client connection negotiation failed: client connection to 127.21.51.129:33587: connect: Connection refused (error 111)
I20260809 16:20:51.579245 885 ts_tablet_manager.cc:1918] T 73ed14def558470da58db9a09aa4d3ad P 2bf53ab985e048d19a823bea592a496d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 16:20:51.579303 885 log.cc:1196] T 73ed14def558470da58db9a09aa4d3ad P 2bf53ab985e048d19a823bea592a496d: Deleting WAL directory at /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292417722232-21710-0/minicluster-data/ts-2/wal/wals/73ed14def558470da58db9a09aa4d3ad
I20260809 16:20:51.579576 885 ts_tablet_manager.cc:1939] T 73ed14def558470da58db9a09aa4d3ad P 2bf53ab985e048d19a823bea592a496d: Deleting consensus metadata
I20260809 16:20:51.579964 32443 catalog_manager.cc:4841] TS 2bf53ab985e048d19a823bea592a496d (127.21.51.131:46809): tablet 73ed14def558470da58db9a09aa4d3ad (table test-table [id=d3aa9f2a6dd340eabc6dd79b7db5a10c]) successfully deleted
I20260809 16:20:51.583088 21710 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrNqUCn/build/release/bin/kudu with pid 504
I20260809 16:20:51.594529 21710 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrNqUCn/build/release/bin/kudu with pid 31980
I20260809 16:20:51.603861 21710 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrNqUCn/build/release/bin/kudu with pid 32270
2026-08-09T16:20:51Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (5355 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786292417722232-21710-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 16:20:51 dist-test-slave-s11t krb5kdc[891](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 16:20:51 dist-test-slave-s11t krb5kdc[891](info): set up 2 sockets
Aug 09 16:20:51 dist-test-slave-s11t krb5kdc[891](info): commencing operation
krb5kdc: starting...
W20260809 16:20:52.861160 21710 mini_kdc.cc:121] Time spent starting KDC: real 1.230s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786292417722232-21710-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786292417722232-21710-0/krb5kdc/test-user.keytab.
Aug 09 16:20:52 dist-test-slave-s11t krb5kdc[891](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292452, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T16:20:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T16:20:52Z Disabled control of system clock
WARNING: no policy specified for oryx/127.21.51.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.21.51.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.21.51.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786292417722232-21710-0/krb5kdc/oryx_127.21.51.190.keytab.
Entry for principal oryx/127.21.51.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786292417722232-21710-0/krb5kdc/oryx_127.21.51.190.keytab.
I20260809 16:20:52.983254 21710 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrNqUCn/build/release/bin/kudu
/tmp/dist-test-taskrNqUCn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786292417722232-21710-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786292417722232-21710-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786292417722232-21710-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786292417722232-21710-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.51.190:37941
--webserver_interface=127.21.51.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786292417722232-21710-0/krb5kdc/oryx_127.21.51.190.keytab
--principal=oryx/127.21.51.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.51.148:40355
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.51.190:37941
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786292417722232-21710-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786292417722232-21710-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786292417722232-21710-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:20:53.067091 910 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:20:53.067334 910 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:20:53.067377 910 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:20:53.068598 910 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 16:20:53.068670 910 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260809 16:20:53.068691 910 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:20:53.068732 910 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:20:53.068770 910 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 16:20:53.068792 910 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 16:20:53.070315 910 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.51.148:40355
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786292417722232-21710-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786292417722232-21710-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.51.190:37941
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786292417722232-21710-0/krb5kdc/oryx_127.21.51.190.keytab
--principal=oryx/127.21.51.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.51.190:37941
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786292417722232-21710-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.51.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.910
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786292417722232-21710-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:20:53.070592 910 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:20:53.070827 910 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:20:53.075817 922 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:20:53.075824 919 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:20:53.075830 917 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:20:53.076148 910 server_base.cc:1034] running on GCE node
I20260809 16:20:53.076406 910 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:20:53.076748 910 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:20:53.077934 910 hybrid_clock.cc:648] HybridClock initialized: now 1786292453077915 us; error 28 us; skew 500 ppm
Aug 09 16:20:53 dist-test-slave-s11t krb5kdc[891](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292453, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.51.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:20:53.080557 910 init.cc:377] Logged in from keytab as oryx/127.21.51.190@KRBTEST.COM (short username oryx)
I20260809 16:20:53.081099 910 webserver.cc:460] Webserver started at http://127.21.51.190:43667/ using document root <none> and password file <none>
I20260809 16:20:53.081333 910 fs_manager.cc:359] Metadata directory not provided
I20260809 16:20:53.081403 910 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:20:53.081538 910 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:20:53.082580 910 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786292417722232-21710-0/minicluster-data/master-0/data/instance:
uuid: "40f8427e400f4dab9038e3273f59f1d0"
format_stamp: "Formatted at 2026-08-09 16:20:53 on dist-test-slave-s11t"
server_key: "47a0fc827872d70100fa665d65c3ffb5"
server_key_iv: "480425a9206a32364828e87f141dadce"
server_key_version: "encryptionkey@0"
I20260809 16:20:53.082990 910 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786292417722232-21710-0/minicluster-data/master-0/wal/instance:
uuid: "40f8427e400f4dab9038e3273f59f1d0"
format_stamp: "Formatted at 2026-08-09 16:20:53 on dist-test-slave-s11t"
server_key: "47a0fc827872d70100fa665d65c3ffb5"
server_key_iv: "480425a9206a32364828e87f141dadce"
server_key_version: "encryptionkey@0"
I20260809 16:20:53.084405 910 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 16:20:53.085255 932 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:20:53.085438 910 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 16:20:53.085538 910 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786292417722232-21710-0/minicluster-data/master-0/data,/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786292417722232-21710-0/minicluster-data/master-0/wal
uuid: "40f8427e400f4dab9038e3273f59f1d0"
format_stamp: "Formatted at 2026-08-09 16:20:53 on dist-test-slave-s11t"
server_key: "47a0fc827872d70100fa665d65c3ffb5"
server_key_iv: "480425a9206a32364828e87f141dadce"
server_key_version: "encryptionkey@0"
I20260809 16:20:53.085626 910 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786292417722232-21710-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786292417722232-21710-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786292417722232-21710-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:20:53.095990 910 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:20:53.097805 910 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:20:53.097967 910 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:20:53.102849 910 rpc_server.cc:307] RPC server started. Bound to: 127.21.51.190:37941
I20260809 16:20:53.102902 1024 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.51.190:37941 every 8 connection(s)
I20260809 16:20:53.103240 910 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786292417722232-21710-0/minicluster-data/master-0/data/info.pb
I20260809 16:20:53.103896 1025 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:20:53.106124 1025 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 40f8427e400f4dab9038e3273f59f1d0: Bootstrap starting.
I20260809 16:20:53.106738 1025 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 40f8427e400f4dab9038e3273f59f1d0: Neither blocks nor log segments found. Creating new log.
I20260809 16:20:53.107055 1025 log.cc:824] T 00000000000000000000000000000000 P 40f8427e400f4dab9038e3273f59f1d0: Log is configured to *not* fsync() on all Append() calls
I20260809 16:20:53.107752 1025 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 40f8427e400f4dab9038e3273f59f1d0: No bootstrap required, opened a new log
I20260809 16:20:53.108392 21710 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrNqUCn/build/release/bin/kudu as pid 910
I20260809 16:20:53.108496 21710 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786292417722232-21710-0/minicluster-data/master-0/wal/instance
I20260809 16:20:53.108719 21710 external_mini_cluster.cc:1428] Setting key 6d8ad6a85258fd2b2ad04c774fe9d59f
I20260809 16:20:53.110412 1025 raft_consensus.cc:348] T 00000000000000000000000000000000 P 40f8427e400f4dab9038e3273f59f1d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "40f8427e400f4dab9038e3273f59f1d0" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 37941 } }
I20260809 16:20:53.110577 1025 raft_consensus.cc:374] T 00000000000000000000000000000000 P 40f8427e400f4dab9038e3273f59f1d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:20:53.110617 1025 raft_consensus.cc:729] T 00000000000000000000000000000000 P 40f8427e400f4dab9038e3273f59f1d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 40f8427e400f4dab9038e3273f59f1d0, State: Initialized, Role: FOLLOWER
I20260809 16:20:53.110809 1025 consensus_queue.cc:260] T 00000000000000000000000000000000 P 40f8427e400f4dab9038e3273f59f1d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "40f8427e400f4dab9038e3273f59f1d0" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 37941 } }
I20260809 16:20:53.110960 1036 raft_consensus.cc:482] T 00000000000000000000000000000000 P 40f8427e400f4dab9038e3273f59f1d0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 16:20:53.111049 1036 raft_consensus.cc:504] T 00000000000000000000000000000000 P 40f8427e400f4dab9038e3273f59f1d0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "40f8427e400f4dab9038e3273f59f1d0" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 37941 } }
I20260809 16:20:53.111159 1036 leader_election.cc:302] T 00000000000000000000000000000000 P 40f8427e400f4dab9038e3273f59f1d0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 40f8427e400f4dab9038e3273f59f1d0; no voters:
I20260809 16:20:53.110939 1025 raft_consensus.cc:388] T 00000000000000000000000000000000 P 40f8427e400f4dab9038e3273f59f1d0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 16:20:53.111253 1025 raft_consensus.cc:482] T 00000000000000000000000000000000 P 40f8427e400f4dab9038e3273f59f1d0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 16:20:53.111299 1025 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 40f8427e400f4dab9038e3273f59f1d0 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:20:53.111340 1036 leader_election.cc:288] T 00000000000000000000000000000000 P 40f8427e400f4dab9038e3273f59f1d0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers
I20260809 16:20:53.112001 1025 raft_consensus.cc:504] T 00000000000000000000000000000000 P 40f8427e400f4dab9038e3273f59f1d0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "40f8427e400f4dab9038e3273f59f1d0" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 37941 } }
I20260809 16:20:53.112085 1025 leader_election.cc:302] T 00000000000000000000000000000000 P 40f8427e400f4dab9038e3273f59f1d0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 40f8427e400f4dab9038e3273f59f1d0; no voters:
I20260809 16:20:53.112118 1036 raft_consensus.cc:2741] T 00000000000000000000000000000000 P 40f8427e400f4dab9038e3273f59f1d0 [term 1 FOLLOWER]: Leader pre-election decision vote started in defunct term 0: won
I20260809 16:20:53.112124 1025 leader_election.cc:288] T 00000000000000000000000000000000 P 40f8427e400f4dab9038e3273f59f1d0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 16:20:53.112262 1037 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 40f8427e400f4dab9038e3273f59f1d0 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:20:53.112345 1037 raft_consensus.cc:686] T 00000000000000000000000000000000 P 40f8427e400f4dab9038e3273f59f1d0 [term 1 LEADER]: Becoming Leader. State: Replica: 40f8427e400f4dab9038e3273f59f1d0, State: Running, Role: LEADER
I20260809 16:20:53.112403 1025 sys_catalog.cc:564] T 00000000000000000000000000000000 P 40f8427e400f4dab9038e3273f59f1d0 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 16:20:53.112517 1037 consensus_queue.cc:237] T 00000000000000000000000000000000 P 40f8427e400f4dab9038e3273f59f1d0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "40f8427e400f4dab9038e3273f59f1d0" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 37941 } }
I20260809 16:20:53.112919 1036 sys_catalog.cc:455] T 00000000000000000000000000000000 P 40f8427e400f4dab9038e3273f59f1d0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "40f8427e400f4dab9038e3273f59f1d0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "40f8427e400f4dab9038e3273f59f1d0" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 37941 } } }
I20260809 16:20:53.112999 1036 sys_catalog.cc:458] T 00000000000000000000000000000000 P 40f8427e400f4dab9038e3273f59f1d0 [sys.catalog]: This master's current role is: LEADER
I20260809 16:20:53.113174 1040 sys_catalog.cc:455] T 00000000000000000000000000000000 P 40f8427e400f4dab9038e3273f59f1d0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 40f8427e400f4dab9038e3273f59f1d0. Latest consensus state: current_term: 1 leader_uuid: "40f8427e400f4dab9038e3273f59f1d0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "40f8427e400f4dab9038e3273f59f1d0" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 37941 } } }
I20260809 16:20:53.113240 1040 sys_catalog.cc:458] T 00000000000000000000000000000000 P 40f8427e400f4dab9038e3273f59f1d0 [sys.catalog]: This master's current role is: LEADER
I20260809 16:20:53.113387 1052 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 16:20:53.114056 1052 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 16:20:53.115641 1052 catalog_manager.cc:1347] Generated new cluster ID: bf0185f670e14164917ea70d970ccfc1
I20260809 16:20:53.115693 1052 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 09 16:20:53 dist-test-slave-s11t krb5kdc[891](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292452, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.21.51.190@KRBTEST.COM
I20260809 16:20:53.121591 1034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:53.109251 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43616 (local address 127.21.51.190:37941)
0809 16:20:53.109437 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0809 16:20:53.109444 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 16:20:53.109469 (+ 25us) server_negotiation.cc:407] Connection header received
0809 16:20:53.110393 (+ 924us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:20:53.110401 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:20:53.110446 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:20:53.110558 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:20:53.111391 (+ 833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:53.112843 (+ 1452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:53.114013 (+ 1170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:53.114843 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:53.114991 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:53.117810 (+ 2819us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:20:53.117814 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:20:53.117814 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:20:53.117827 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:20:53.119904 (+ 2077us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:20:53.120296 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:20:53.120299 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:20:53.120299 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:20:53.120435 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:20:53.120648 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:20:53.120649 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:20:53.120650 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:20:53.120973 (+ 323us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:20:53.121137 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0809 16:20:53.121239 (+ 102us) server_negotiation.cc:299] Negotiation successful
0809 16:20:53.121397 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":134,"thread_start_us":88,"threads_started":1}
I20260809 16:20:53.122025 1052 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 16:20:53.122607 1052 catalog_manager.cc:1504] Loading token signing keys...
I20260809 16:20:53.124501 1052 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 40f8427e400f4dab9038e3273f59f1d0: Generated new TSK 0
I20260809 16:20:53.124665 1052 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.21.51.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.21.51.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.21.51.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786292417722232-21710-0/krb5kdc/oryx_127.21.51.129.keytab.
Entry for principal oryx/127.21.51.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786292417722232-21710-0/krb5kdc/oryx_127.21.51.129.keytab.
I20260809 16:20:53.160830 21710 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrNqUCn/build/release/bin/kudu
/tmp/dist-test-taskrNqUCn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786292417722232-21710-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786292417722232-21710-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786292417722232-21710-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786292417722232-21710-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.51.129:0
--local_ip_for_outbound_sockets=127.21.51.129
--webserver_interface=127.21.51.129
--webserver_port=0
--tserver_master_addrs=127.21.51.190:37941
--keytab_file=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786292417722232-21710-0/krb5kdc/oryx_127.21.51.129.keytab
--principal=oryx/127.21.51.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.51.148:40355
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786292417722232-21710-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786292417722232-21710-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786292417722232-21710-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:20:53.243469 1065 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:20:53.243707 1065 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:20:53.243750 1065 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:20:53.245041 1065 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:20:53.245106 1065 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:20:53.245141 1065 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 16:20:53.245183 1065 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.51.129
I20260809 16:20:53.246703 1065 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.51.148:40355
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786292417722232-21710-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786292417722232-21710-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786292417722232-21710-0/krb5kdc/oryx_127.21.51.129.keytab
--principal=oryx/127.21.51.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.51.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786292417722232-21710-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.51.129
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.21.51.190:37941
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1065
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.51.129
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786292417722232-21710-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:20:53.246971 1065 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:20:53.247207 1065 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 16:20:53.252389 1065 server_base.cc:1034] running on GCE node
W20260809 16:20:53.252350 1075 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:20:53.252365 1072 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:20:53.252362 1073 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:20:53.252777 1065 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:20:53.253055 1065 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:20:53.254206 1065 hybrid_clock.cc:648] HybridClock initialized: now 1786292453254193 us; error 29 us; skew 500 ppm
Aug 09 16:20:53 dist-test-slave-s11t krb5kdc[891](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292453, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.51.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:20:53.256718 1065 init.cc:377] Logged in from keytab as oryx/127.21.51.129@KRBTEST.COM (short username oryx)
I20260809 16:20:53.257143 1065 webserver.cc:460] Webserver started at http://127.21.51.129:42373/ using document root <none> and password file <none>
I20260809 16:20:53.257400 1065 fs_manager.cc:359] Metadata directory not provided
I20260809 16:20:53.257465 1065 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:20:53.257597 1065 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:20:53.258607 1065 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786292417722232-21710-0/minicluster-data/ts-0/data/instance:
uuid: "fab00ea5697f40b8a24b8c4fdc8084e0"
format_stamp: "Formatted at 2026-08-09 16:20:53 on dist-test-slave-s11t"
server_key: "dfbc31c94c085b524921f830b6e76f27"
server_key_iv: "3dffbbed3c45ed50fc21f90666ce759b"
server_key_version: "encryptionkey@0"
I20260809 16:20:53.258976 1065 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786292417722232-21710-0/minicluster-data/ts-0/wal/instance:
uuid: "fab00ea5697f40b8a24b8c4fdc8084e0"
format_stamp: "Formatted at 2026-08-09 16:20:53 on dist-test-slave-s11t"
server_key: "dfbc31c94c085b524921f830b6e76f27"
server_key_iv: "3dffbbed3c45ed50fc21f90666ce759b"
server_key_version: "encryptionkey@0"
I20260809 16:20:53.260305 1065 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 16:20:53.261109 1084 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:20:53.261301 1065 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260809 16:20:53.261420 1065 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786292417722232-21710-0/minicluster-data/ts-0/data,/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786292417722232-21710-0/minicluster-data/ts-0/wal
uuid: "fab00ea5697f40b8a24b8c4fdc8084e0"
format_stamp: "Formatted at 2026-08-09 16:20:53 on dist-test-slave-s11t"
server_key: "dfbc31c94c085b524921f830b6e76f27"
server_key_iv: "3dffbbed3c45ed50fc21f90666ce759b"
server_key_version: "encryptionkey@0"
I20260809 16:20:53.261492 1065 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786292417722232-21710-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786292417722232-21710-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786292417722232-21710-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:20:53.270231 1065 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:20:53.271943 1065 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:20:53.272109 1065 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:20:53.272640 1065 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:20:53.272720 1065 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:20:53.272774 1065 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:20:53.272807 1065 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 16:20:53 dist-test-slave-s11t krb5kdc[891](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292453, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.51.129@KRBTEST.COM for oryx/127.21.51.190@KRBTEST.COM
I20260809 16:20:53.279798 1065 rpc_server.cc:307] RPC server started. Bound to: 127.21.51.129:33285
I20260809 16:20:53.280042 1267 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.51.129:33285 every 8 connection(s)
I20260809 16:20:53.280215 1065 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786292417722232-21710-0/minicluster-data/ts-0/data/info.pb
I20260809 16:20:53.283224 1034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:53.273115 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.51.129:46677 (local address 127.21.51.190:37941)
0809 16:20:53.273172 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0809 16:20:53.273174 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:20:53.273407 (+ 233us) server_negotiation.cc:407] Connection header received
0809 16:20:53.273892 (+ 485us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:20:53.273893 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:20:53.273910 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:20:53.273966 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:20:53.274784 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:53.275635 (+ 851us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:53.277183 (+ 1548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:53.277432 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:53.277507 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:53.279774 (+ 2267us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:20:53.279776 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:20:53.279776 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:20:53.279785 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:20:53.281764 (+ 1979us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:20:53.282195 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:20:53.282196 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:20:53.282197 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:20:53.282256 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:20:53.282574 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:20:53.282574 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:20:53.282575 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:20:53.282754 (+ 179us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:20:53.282825 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0809 16:20:53.283090 (+ 265us) server_negotiation.cc:299] Negotiation successful
0809 16:20:53.283161 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260809 16:20:53.283352 1099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:53.273095 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:37941 (local address 127.21.51.129:46677)
0809 16:20:53.273252 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0809 16:20:53.273285 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0809 16:20:53.273780 (+ 495us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:20:53.274031 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:20:53.274035 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:20:53.274393 (+ 358us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:20:53.274662 (+ 269us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:20:53.274667 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:20:53.275748 (+ 1081us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:20:53.275749 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:20:53.277086 (+ 1337us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:20:53.277089 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:20:53.277542 (+ 453us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:20:53.277543 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:20:53.277676 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:53.278182 (+ 506us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:20:53.278189 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:20:53.279652 (+ 1463us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:20:53.281860 (+ 2208us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:20:53.281867 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:20:53.281870 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:20:53.282110 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:20:53.282366 (+ 256us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:20:53.282367 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:20:53.282368 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:20:53.282508 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:20:53.282835 (+ 327us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:20:53.282849 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:20:53.282962 (+ 113us) client_negotiation.cc:769] Sending connection context
0809 16:20:53.283055 (+ 93us) client_negotiation.cc:240] Negotiation successful
0809 16:20:53.283175 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":80,"threads_started":1}
I20260809 16:20:53.283835 1269 heartbeater.cc:347] Connected to a master server at 127.21.51.190:37941
I20260809 16:20:53.283949 1269 heartbeater.cc:464] Registering TS with master...
I20260809 16:20:53.284143 1269 heartbeater.cc:511] Master 127.21.51.190:37941 requested a full tablet report, sending...
I20260809 16:20:53.284687 954 ts_manager.cc:194] Registered new tserver with Master: fab00ea5697f40b8a24b8c4fdc8084e0 (127.21.51.129:33285)
I20260809 16:20:53.285431 954 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.21.51.129@KRBTEST.COM'} at 127.21.51.129:46677
I20260809 16:20:53.286367 21710 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrNqUCn/build/release/bin/kudu as pid 1065
I20260809 16:20:53.286479 21710 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786292417722232-21710-0/minicluster-data/ts-0/wal/instance
I20260809 16:20:53.286684 21710 external_mini_cluster.cc:1428] Setting key f5961be366227178630bd21a9ccd450d
I20260809 16:20:53.291426 1279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:53.284890 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:37941 (local address 127.21.51.129:55309)
0809 16:20:53.285616 (+ 726us) negotiation.cc:107] Waiting for socket to connect
0809 16:20:53.285626 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 16:20:53.285710 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:20:53.285849 (+ 139us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:20:53.285850 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:20:53.286026 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:20:53.286080 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:20:53.286082 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:20:53.287011 (+ 929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:20:53.287012 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:20:53.287758 (+ 746us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:20:53.287760 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:20:53.288225 (+ 465us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:20:53.288226 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:20:53.288286 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:53.288739 (+ 453us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:20:53.288743 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:20:53.289019 (+ 276us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:20:53.290594 (+ 1575us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:20:53.290594 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:20:53.290595 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:20:53.290753 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:20:53.290951 (+ 198us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:20:53.290951 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:20:53.290951 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:20:53.290994 (+ 43us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:20:53.291222 (+ 228us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:20:53.291223 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:20:53.291254 (+ 31us) client_negotiation.cc:769] Sending connection context
0809 16:20:53.291293 (+ 39us) client_negotiation.cc:240] Negotiation successful
0809 16:20:53.291365 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":705,"thread_start_us":81,"threads_started":1}
I20260809 16:20:53.291558 1034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:53.284955 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.51.129:55309 (local address 127.21.51.190:37941)
0809 16:20:53.284992 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0809 16:20:53.284994 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:20:53.285698 (+ 704us) server_negotiation.cc:407] Connection header received
0809 16:20:53.285769 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:20:53.285770 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:20:53.285781 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:20:53.285824 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:20:53.286157 (+ 333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:53.286882 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:53.287842 (+ 960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:53.288131 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:53.288242 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:53.289096 (+ 854us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:20:53.289097 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:20:53.289098 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:20:53.289105 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:20:53.290527 (+ 1422us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:20:53.290823 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:20:53.290825 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:20:53.290825 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:20:53.290887 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:20:53.291030 (+ 143us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:20:53.291030 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:20:53.291031 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:20:53.291166 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:20:53.291236 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0809 16:20:53.291327 (+ 91us) server_negotiation.cc:299] Negotiation successful
0809 16:20:53.291426 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
WARNING: no policy specified for oryx/127.21.51.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.21.51.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.21.51.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786292417722232-21710-0/krb5kdc/oryx_127.21.51.130.keytab.
Entry for principal oryx/127.21.51.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786292417722232-21710-0/krb5kdc/oryx_127.21.51.130.keytab.
I20260809 16:20:53.313620 21710 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrNqUCn/build/release/bin/kudu
/tmp/dist-test-taskrNqUCn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786292417722232-21710-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786292417722232-21710-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786292417722232-21710-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786292417722232-21710-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.51.130:0
--local_ip_for_outbound_sockets=127.21.51.130
--webserver_interface=127.21.51.130
--webserver_port=0
--tserver_master_addrs=127.21.51.190:37941
--keytab_file=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786292417722232-21710-0/krb5kdc/oryx_127.21.51.130.keytab
--principal=oryx/127.21.51.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.51.148:40355
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786292417722232-21710-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786292417722232-21710-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786292417722232-21710-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:20:53.395066 1283 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:20:53.395279 1283 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:20:53.395319 1283 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:20:53.396478 1283 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:20:53.396544 1283 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:20:53.396579 1283 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 16:20:53.396637 1283 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.51.130
I20260809 16:20:53.398171 1283 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.51.148:40355
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786292417722232-21710-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786292417722232-21710-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786292417722232-21710-0/krb5kdc/oryx_127.21.51.130.keytab
--principal=oryx/127.21.51.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.51.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786292417722232-21710-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.51.130
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.21.51.190:37941
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1283
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.51.130
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786292417722232-21710-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:20:53.398442 1283 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:20:53.398664 1283 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:20:53.403641 1291 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:20:53.403653 1295 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:20:53.403654 1292 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:20:53.403963 1283 server_base.cc:1034] running on GCE node
I20260809 16:20:53.404194 1283 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:20:53.404433 1283 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:20:53.405606 1283 hybrid_clock.cc:648] HybridClock initialized: now 1786292453405602 us; error 40 us; skew 500 ppm
Aug 09 16:20:53 dist-test-slave-s11t krb5kdc[891](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292453, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.51.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:20:53.408233 1283 init.cc:377] Logged in from keytab as oryx/127.21.51.130@KRBTEST.COM (short username oryx)
I20260809 16:20:53.408668 1283 webserver.cc:460] Webserver started at http://127.21.51.130:42919/ using document root <none> and password file <none>
I20260809 16:20:53.408934 1283 fs_manager.cc:359] Metadata directory not provided
I20260809 16:20:53.409005 1283 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:20:53.409148 1283 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:20:53.410167 1283 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786292417722232-21710-0/minicluster-data/ts-1/data/instance:
uuid: "004c79d477944649ab47d6f916e125fd"
format_stamp: "Formatted at 2026-08-09 16:20:53 on dist-test-slave-s11t"
server_key: "bf7482f620292f8a1aad19511c4707ed"
server_key_iv: "af52ec3e1e25c8124cc9ea626e767a14"
server_key_version: "encryptionkey@0"
I20260809 16:20:53.410549 1283 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786292417722232-21710-0/minicluster-data/ts-1/wal/instance:
uuid: "004c79d477944649ab47d6f916e125fd"
format_stamp: "Formatted at 2026-08-09 16:20:53 on dist-test-slave-s11t"
server_key: "bf7482f620292f8a1aad19511c4707ed"
server_key_iv: "af52ec3e1e25c8124cc9ea626e767a14"
server_key_version: "encryptionkey@0"
I20260809 16:20:53.411959 1283 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 16:20:53.412667 1304 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:20:53.412984 1283 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 16:20:53.413086 1283 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786292417722232-21710-0/minicluster-data/ts-1/data,/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786292417722232-21710-0/minicluster-data/ts-1/wal
uuid: "004c79d477944649ab47d6f916e125fd"
format_stamp: "Formatted at 2026-08-09 16:20:53 on dist-test-slave-s11t"
server_key: "bf7482f620292f8a1aad19511c4707ed"
server_key_iv: "af52ec3e1e25c8124cc9ea626e767a14"
server_key_version: "encryptionkey@0"
I20260809 16:20:53.413174 1283 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786292417722232-21710-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786292417722232-21710-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786292417722232-21710-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:20:53.442255 1283 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:20:53.444485 1283 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:20:53.444665 1283 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:20:53.445225 1283 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:20:53.445293 1283 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:20:53.445340 1283 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:20:53.445389 1283 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 16:20:53 dist-test-slave-s11t krb5kdc[891](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292453, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.51.130@KRBTEST.COM for oryx/127.21.51.190@KRBTEST.COM
I20260809 16:20:53.452803 1283 rpc_server.cc:307] RPC server started. Bound to: 127.21.51.130:39867
I20260809 16:20:53.452905 1484 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.51.130:39867 every 8 connection(s)
I20260809 16:20:53.453259 1283 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786292417722232-21710-0/minicluster-data/ts-1/data/info.pb
I20260809 16:20:53.455972 1034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:53.445780 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.51.130:47785 (local address 127.21.51.190:37941)
0809 16:20:53.445852 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0809 16:20:53.445854 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:20:53.446134 (+ 280us) server_negotiation.cc:407] Connection header received
0809 16:20:53.446853 (+ 719us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:20:53.446854 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:20:53.446865 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:20:53.446915 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:20:53.447829 (+ 914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:53.448654 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:53.450350 (+ 1696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:53.450620 (+ 270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:53.450704 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:53.453025 (+ 2321us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:20:53.453026 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:20:53.453027 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:20:53.453035 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:20:53.454684 (+ 1649us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:20:53.455171 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:20:53.455172 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:20:53.455172 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:20:53.455233 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:20:53.455484 (+ 251us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:20:53.455485 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:20:53.455485 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:20:53.455593 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:20:53.455643 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0809 16:20:53.455832 (+ 189us) server_negotiation.cc:299] Negotiation successful
0809 16:20:53.455908 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260809 16:20:53.456130 1318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:53.445725 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:37941 (local address 127.21.51.130:47785)
0809 16:20:53.445995 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0809 16:20:53.446038 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0809 16:20:53.446756 (+ 718us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:20:53.446992 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:20:53.446996 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:20:53.447419 (+ 423us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:20:53.447727 (+ 308us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:20:53.447733 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:20:53.448785 (+ 1052us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:20:53.448786 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:20:53.450252 (+ 1466us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:20:53.450256 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:20:53.450712 (+ 456us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:20:53.450713 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:20:53.450868 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:53.451384 (+ 516us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:20:53.451400 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:20:53.452927 (+ 1527us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:20:53.454774 (+ 1847us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:20:53.454780 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:20:53.454781 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:20:53.455058 (+ 277us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:20:53.455304 (+ 246us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:20:53.455305 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:20:53.455305 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:20:53.455421 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:20:53.455640 (+ 219us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:20:53.455642 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:20:53.455744 (+ 102us) client_negotiation.cc:769] Sending connection context
0809 16:20:53.455819 (+ 75us) client_negotiation.cc:240] Negotiation successful
0809 16:20:53.455938 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":231,"thread_start_us":77,"threads_started":1}
I20260809 16:20:53.456650 1486 heartbeater.cc:347] Connected to a master server at 127.21.51.190:37941
I20260809 16:20:53.456784 1486 heartbeater.cc:464] Registering TS with master...
I20260809 16:20:53.456971 1486 heartbeater.cc:511] Master 127.21.51.190:37941 requested a full tablet report, sending...
I20260809 16:20:53.457414 955 ts_manager.cc:194] Registered new tserver with Master: 004c79d477944649ab47d6f916e125fd (127.21.51.130:39867)
I20260809 16:20:53.458139 955 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.21.51.130@KRBTEST.COM'} at 127.21.51.130:47785
I20260809 16:20:53.458770 21710 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrNqUCn/build/release/bin/kudu as pid 1283
I20260809 16:20:53.458904 21710 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786292417722232-21710-0/minicluster-data/ts-1/wal/instance
I20260809 16:20:53.459134 21710 external_mini_cluster.cc:1428] Setting key 955ea8dc0a0305a03087337b366d2dc7
I20260809 16:20:53.464213 1034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:53.457772 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.51.130:56923 (local address 127.21.51.190:37941)
0809 16:20:53.457813 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0809 16:20:53.457815 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:20:53.457897 (+ 82us) server_negotiation.cc:407] Connection header received
0809 16:20:53.457972 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:20:53.457973 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:20:53.457983 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:20:53.458043 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:20:53.458406 (+ 363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:53.459057 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:53.460101 (+ 1044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:53.460302 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:53.460354 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:53.461477 (+ 1123us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:20:53.461478 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:20:53.461478 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:20:53.461485 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:20:53.462974 (+ 1489us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:20:53.463433 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:20:53.463434 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:20:53.463434 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:20:53.463488 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:20:53.463692 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:20:53.463693 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:20:53.463693 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:20:53.463814 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:20:53.463861 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0809 16:20:53.464038 (+ 177us) server_negotiation.cc:299] Negotiation successful
0809 16:20:53.464136 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260809 16:20:53.464411 1497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:53.457691 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:37941 (local address 127.21.51.130:56923)
0809 16:20:53.457831 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0809 16:20:53.457843 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 16:20:53.457908 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:20:53.458060 (+ 152us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:20:53.458061 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:20:53.458272 (+ 211us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:20:53.458336 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:20:53.458338 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:20:53.459161 (+ 823us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:20:53.459162 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:20:53.460011 (+ 849us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:20:53.460014 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:20:53.460379 (+ 365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:20:53.460380 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:20:53.460454 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:53.460984 (+ 530us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:20:53.460990 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:20:53.461388 (+ 398us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:20:53.463084 (+ 1696us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:20:53.463085 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:20:53.463086 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:20:53.463350 (+ 264us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:20:53.463556 (+ 206us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:20:53.463557 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:20:53.463557 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:20:53.463629 (+ 72us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:20:53.463876 (+ 247us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:20:53.463877 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:20:53.463932 (+ 55us) client_negotiation.cc:769] Sending connection context
0809 16:20:53.464024 (+ 92us) client_negotiation.cc:240] Negotiation successful
0809 16:20:53.464187 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":75,"threads_started":1}
WARNING: no policy specified for oryx/127.21.51.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.21.51.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.21.51.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786292417722232-21710-0/krb5kdc/oryx_127.21.51.131.keytab.
Entry for principal oryx/127.21.51.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786292417722232-21710-0/krb5kdc/oryx_127.21.51.131.keytab.
I20260809 16:20:53.489149 21710 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrNqUCn/build/release/bin/kudu
/tmp/dist-test-taskrNqUCn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786292417722232-21710-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786292417722232-21710-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786292417722232-21710-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786292417722232-21710-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.51.131:0
--local_ip_for_outbound_sockets=127.21.51.131
--webserver_interface=127.21.51.131
--webserver_port=0
--tserver_master_addrs=127.21.51.190:37941
--keytab_file=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786292417722232-21710-0/krb5kdc/oryx_127.21.51.131.keytab
--principal=oryx/127.21.51.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.51.148:40355
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786292417722232-21710-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786292417722232-21710-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786292417722232-21710-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:20:53.573359 1502 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:20:53.573549 1502 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:20:53.573572 1502 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:20:53.574646 1502 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:20:53.574682 1502 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:20:53.574702 1502 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 16:20:53.574719 1502 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.51.131
I20260809 16:20:53.576124 1502 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.51.148:40355
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786292417722232-21710-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786292417722232-21710-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786292417722232-21710-0/krb5kdc/oryx_127.21.51.131.keytab
--principal=oryx/127.21.51.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.51.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786292417722232-21710-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.51.131
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.21.51.190:37941
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1502
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.51.131
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786292417722232-21710-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:20:53.576299 1502 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:20:53.576474 1502 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:20:53.581369 1512 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:20:53.581393 1502 server_base.cc:1034] running on GCE node
W20260809 16:20:53.581349 1517 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:20:53.581394 1509 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:20:53.581784 1502 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:20:53.582039 1502 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:20:53.583194 1502 hybrid_clock.cc:648] HybridClock initialized: now 1786292453583187 us; error 35 us; skew 500 ppm
Aug 09 16:20:53 dist-test-slave-s11t krb5kdc[891](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292453, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.51.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:20:53.585855 1502 init.cc:377] Logged in from keytab as oryx/127.21.51.131@KRBTEST.COM (short username oryx)
I20260809 16:20:53.586311 1502 webserver.cc:460] Webserver started at http://127.21.51.131:38307/ using document root <none> and password file <none>
I20260809 16:20:53.586529 1502 fs_manager.cc:359] Metadata directory not provided
I20260809 16:20:53.586594 1502 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:20:53.586745 1502 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:20:53.587793 1502 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786292417722232-21710-0/minicluster-data/ts-2/data/instance:
uuid: "9525d3dbf73f46fa93cf72ad9cce21f2"
format_stamp: "Formatted at 2026-08-09 16:20:53 on dist-test-slave-s11t"
server_key: "76bcbb926a2ecf2c07703ebb316e645b"
server_key_iv: "5eed8ef869ea0dfae8293f36c67b31fb"
server_key_version: "encryptionkey@0"
I20260809 16:20:53.588155 1502 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786292417722232-21710-0/minicluster-data/ts-2/wal/instance:
uuid: "9525d3dbf73f46fa93cf72ad9cce21f2"
format_stamp: "Formatted at 2026-08-09 16:20:53 on dist-test-slave-s11t"
server_key: "76bcbb926a2ecf2c07703ebb316e645b"
server_key_iv: "5eed8ef869ea0dfae8293f36c67b31fb"
server_key_version: "encryptionkey@0"
I20260809 16:20:53.589584 1502 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 16:20:53.590309 1533 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:20:53.590493 1502 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 16:20:53.590592 1502 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786292417722232-21710-0/minicluster-data/ts-2/data,/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786292417722232-21710-0/minicluster-data/ts-2/wal
uuid: "9525d3dbf73f46fa93cf72ad9cce21f2"
format_stamp: "Formatted at 2026-08-09 16:20:53 on dist-test-slave-s11t"
server_key: "76bcbb926a2ecf2c07703ebb316e645b"
server_key_iv: "5eed8ef869ea0dfae8293f36c67b31fb"
server_key_version: "encryptionkey@0"
I20260809 16:20:53.590659 1502 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786292417722232-21710-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786292417722232-21710-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786292417722232-21710-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:20:53.600003 1502 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:20:53.602221 1502 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:20:53.602381 1502 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:20:53.602852 1502 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:20:53.602916 1502 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:20:53.602958 1502 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:20:53.602998 1502 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:20:53.609476 1502 rpc_server.cc:307] RPC server started. Bound to: 127.21.51.131:41441
I20260809 16:20:53.609532 1706 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.51.131:41441 every 8 connection(s)
I20260809 16:20:53.609879 1502 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786292417722232-21710-0/minicluster-data/ts-2/data/info.pb
Aug 09 16:20:53 dist-test-slave-s11t krb5kdc[891](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292453, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.51.131@KRBTEST.COM for oryx/127.21.51.190@KRBTEST.COM
I20260809 16:20:53.613041 1034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:53.603381 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.51.131:44625 (local address 127.21.51.190:37941)
0809 16:20:53.603457 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0809 16:20:53.603460 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 16:20:53.603648 (+ 188us) server_negotiation.cc:407] Connection header received
0809 16:20:53.604080 (+ 432us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:20:53.604082 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:20:53.604096 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:20:53.604181 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:20:53.605031 (+ 850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:53.605905 (+ 874us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:53.607455 (+ 1550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:53.607792 (+ 337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:53.607881 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:53.610544 (+ 2663us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:20:53.610545 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:20:53.610546 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:20:53.610552 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:20:53.611791 (+ 1239us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:20:53.612156 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:20:53.612157 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:20:53.612157 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:20:53.612207 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:20:53.612434 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:20:53.612435 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:20:53.612435 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:20:53.612538 (+ 103us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:20:53.612593 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0809 16:20:53.612909 (+ 316us) server_negotiation.cc:299] Negotiation successful
0809 16:20:53.612976 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260809 16:20:53.613221 1549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:53.603357 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:37941 (local address 127.21.51.131:44625)
0809 16:20:53.603525 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0809 16:20:53.603567 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0809 16:20:53.603992 (+ 425us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:20:53.604257 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:20:53.604261 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:20:53.604619 (+ 358us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:20:53.604928 (+ 309us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:20:53.604935 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:20:53.606003 (+ 1068us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:20:53.606004 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:20:53.607355 (+ 1351us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:20:53.607359 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:20:53.607901 (+ 542us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:20:53.607902 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:20:53.608006 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:53.608556 (+ 550us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:20:53.608568 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:20:53.610460 (+ 1892us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:20:53.611872 (+ 1412us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:20:53.611877 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:20:53.611879 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:20:53.612091 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:20:53.612270 (+ 179us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:20:53.612272 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:20:53.612272 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:20:53.612378 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:20:53.612595 (+ 217us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:20:53.612599 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:20:53.612786 (+ 187us) client_negotiation.cc:769] Sending connection context
0809 16:20:53.612878 (+ 92us) client_negotiation.cc:240] Negotiation successful
0809 16:20:53.612995 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":87,"threads_started":1}
I20260809 16:20:53.613703 1707 heartbeater.cc:347] Connected to a master server at 127.21.51.190:37941
I20260809 16:20:53.613790 1707 heartbeater.cc:464] Registering TS with master...
I20260809 16:20:53.613965 1707 heartbeater.cc:511] Master 127.21.51.190:37941 requested a full tablet report, sending...
I20260809 16:20:53.614418 954 ts_manager.cc:194] Registered new tserver with Master: 9525d3dbf73f46fa93cf72ad9cce21f2 (127.21.51.131:41441)
I20260809 16:20:53.614986 954 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.21.51.131@KRBTEST.COM'} at 127.21.51.131:44625
I20260809 16:20:53.615500 21710 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrNqUCn/build/release/bin/kudu as pid 1502
I20260809 16:20:53.615643 21710 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786292417722232-21710-0/minicluster-data/ts-2/wal/instance
I20260809 16:20:53.615859 21710 external_mini_cluster.cc:1428] Setting key 5c9691b84004e5062d5a14911b444e71
I20260809 16:20:53.616676 21710 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 16:20:53.620975 1034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:53.614962 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.51.131:49397 (local address 127.21.51.190:37941)
0809 16:20:53.615006 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0809 16:20:53.615007 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 16:20:53.615249 (+ 242us) server_negotiation.cc:407] Connection header received
0809 16:20:53.615347 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:20:53.615347 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:20:53.615358 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:20:53.615417 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:20:53.615843 (+ 426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:53.616544 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:53.617376 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:53.617651 (+ 275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:53.617745 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:53.618570 (+ 825us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:20:53.618571 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:20:53.618572 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:20:53.618579 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:20:53.619854 (+ 1275us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:20:53.620255 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:20:53.620256 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:20:53.620256 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:20:53.620312 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:20:53.620519 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:20:53.620519 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:20:53.620519 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:20:53.620613 (+ 94us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:20:53.620662 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0809 16:20:53.620853 (+ 191us) server_negotiation.cc:299] Negotiation successful
0809 16:20:53.620917 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260809 16:20:53.621085 1720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:53.614812 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:37941 (local address 127.21.51.131:49397)
0809 16:20:53.615155 (+ 343us) negotiation.cc:107] Waiting for socket to connect
0809 16:20:53.615179 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0809 16:20:53.615261 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:20:53.615428 (+ 167us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:20:53.615429 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:20:53.615633 (+ 204us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:20:53.615731 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:20:53.615734 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:20:53.616726 (+ 992us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:20:53.616727 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:20:53.617295 (+ 568us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:20:53.617297 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:20:53.617741 (+ 444us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:20:53.617741 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:20:53.617789 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:53.618200 (+ 411us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:20:53.618205 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:20:53.618490 (+ 285us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:20:53.619939 (+ 1449us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:20:53.619940 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:20:53.619941 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:20:53.620176 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:20:53.620381 (+ 205us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:20:53.620382 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:20:53.620382 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:20:53.620456 (+ 74us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:20:53.620673 (+ 217us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:20:53.620674 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:20:53.620746 (+ 72us) client_negotiation.cc:769] Sending connection context
0809 16:20:53.620820 (+ 74us) client_negotiation.cc:240] Negotiation successful
0809 16:20:53.620937 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":316,"thread_start_us":76,"threads_started":1}
Aug 09 16:20:53 dist-test-slave-s11t krb5kdc[891](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292453, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 09 16:20:53 dist-test-slave-s11t krb5kdc[891](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.21.51.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 16:20:53 dist-test-slave-s11t krb5kdc[891](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.21.51.190@KRBTEST.COM, Server not found in Kerberos database
W20260809 16:20:53.640439 1034 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:20:53.634911 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43630 (local address 127.21.51.190:37941)
0809 16:20:53.634971 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0809 16:20:53.634973 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:20:53.634987 (+ 14us) server_negotiation.cc:407] Connection header received
0809 16:20:53.635013 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:20:53.635014 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:20:53.635023 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:20:53.635074 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:20:53.635661 (+ 587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:53.636172 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:53.637000 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:53.637190 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:53.637253 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:53.640352 (+ 3099us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43630: BlockingRecv error: recv got EOF from 127.0.0.1:43630 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
Aug 09 16:20:53 dist-test-slave-s11t krb5kdc[891](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292453, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.21.51.190@KRBTEST.COM
I20260809 16:20:53.647917 1034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:53.641168 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43632 (local address 127.21.51.190:37941)
0809 16:20:53.641224 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0809 16:20:53.641226 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:20:53.641329 (+ 103us) server_negotiation.cc:407] Connection header received
0809 16:20:53.641359 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:20:53.641360 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:20:53.641370 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:20:53.641429 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:20:53.641853 (+ 424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:53.642345 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:53.643012 (+ 667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:53.643191 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:53.643249 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:53.645446 (+ 2197us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:20:53.645452 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:20:53.645452 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:20:53.645458 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:20:53.646739 (+ 1281us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:20:53.647110 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:20:53.647111 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:20:53.647111 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:20:53.647173 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:20:53.647375 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:20:53.647376 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:20:53.647376 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:20:53.647486 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:20:53.647585 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0809 16:20:53.647698 (+ 113us) server_negotiation.cc:299] Negotiation successful
0809 16:20:53.647775 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260809 16:20:53.649585 954 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43632:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 16:20:53.651173 954 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 16:20:53.658206 1754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:53.653971 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47044 (local address 127.21.51.130:39867)
0809 16:20:53.654267 (+ 296us) server_negotiation.cc:206] Beginning negotiation
0809 16:20:53.654271 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 16:20:53.654280 (+ 9us) server_negotiation.cc:407] Connection header received
0809 16:20:53.654322 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:20:53.654324 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:20:53.654340 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:20:53.654616 (+ 276us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:20:53.654682 (+ 66us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:53.655772 (+ 1090us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:53.657124 (+ 1352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:53.657697 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:53.657788 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:53.657869 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0809 16:20:53.657996 (+ 127us) server_negotiation.cc:299] Negotiation successful
0809 16:20:53.658037 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":269,"thread_start_us":93,"threads_started":1}
I20260809 16:20:53.658272 1750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:53.653773 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.131:41441 (local address 127.0.0.1:37256)
0809 16:20:53.654211 (+ 438us) negotiation.cc:107] Waiting for socket to connect
0809 16:20:53.654221 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 16:20:53.654262 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:20:53.654706 (+ 444us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:20:53.654707 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:20:53.654709 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:20:53.654759 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:20:53.654760 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:20:53.655607 (+ 847us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:20:53.655608 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:20:53.656977 (+ 1369us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:20:53.656980 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:20:53.658038 (+ 1058us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:20:53.658039 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:20:53.658098 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:53.658102 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 16:20:53.658152 (+ 50us) client_negotiation.cc:240] Negotiation successful
0809 16:20:53.658191 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":419,"mutex_wait_us":2,"thread_start_us":118,"threads_started":1}
I20260809 16:20:53.658506 1747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:53.653772 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.130:39867 (local address 127.0.0.1:47044)
0809 16:20:53.653990 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0809 16:20:53.654033 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0809 16:20:53.654102 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:20:53.654468 (+ 366us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:20:53.654471 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:20:53.654473 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:20:53.654535 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:20:53.654537 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:20:53.655874 (+ 1337us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:20:53.655875 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:20:53.657060 (+ 1185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:20:53.657063 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:20:53.657792 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:20:53.657794 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:20:53.657890 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:53.657905 (+ 15us) client_negotiation.cc:769] Sending connection context
0809 16:20:53.657980 (+ 75us) client_negotiation.cc:240] Negotiation successful
0809 16:20:53.658206 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":60,"threads_started":1}
I20260809 16:20:53.658545 1755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:53.654046 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37256 (local address 127.21.51.131:41441)
0809 16:20:53.654398 (+ 352us) server_negotiation.cc:206] Beginning negotiation
0809 16:20:53.654403 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 16:20:53.654414 (+ 11us) server_negotiation.cc:407] Connection header received
0809 16:20:53.654531 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:20:53.654534 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:20:53.654553 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:20:53.654629 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:20:53.654830 (+ 201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:53.655520 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:53.657068 (+ 1548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:53.657857 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:53.657920 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:53.658004 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0809 16:20:53.658404 (+ 400us) server_negotiation.cc:299] Negotiation successful
0809 16:20:53.658451 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":298,"thread_start_us":82,"threads_started":1}
I20260809 16:20:53.659086 1746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:53.653794 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.129:33285 (local address 127.0.0.1:39410)
0809 16:20:53.654211 (+ 417us) negotiation.cc:107] Waiting for socket to connect
0809 16:20:53.654221 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 16:20:53.654277 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:20:53.654530 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:20:53.654531 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:20:53.654532 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:20:53.654574 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:20:53.654575 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:20:53.656750 (+ 2175us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:20:53.656751 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:20:53.657779 (+ 1028us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:20:53.657783 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:20:53.658847 (+ 1064us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:20:53.658848 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:20:53.658918 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:53.658923 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 16:20:53.658988 (+ 65us) client_negotiation.cc:240] Negotiation successful
0809 16:20:53.659026 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":405,"thread_start_us":91,"threads_started":1}
I20260809 16:20:53.659102 1756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:53.654061 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39410 (local address 127.21.51.129:33285)
0809 16:20:53.654221 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0809 16:20:53.654226 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 16:20:53.654269 (+ 43us) server_negotiation.cc:407] Connection header received
0809 16:20:53.654343 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:20:53.654345 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:20:53.654358 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:20:53.654457 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:20:53.654651 (+ 194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:53.655763 (+ 1112us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:53.657891 (+ 2128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:53.658764 (+ 873us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:53.658846 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:53.658925 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0809 16:20:53.659005 (+ 80us) server_negotiation.cc:299] Negotiation successful
0809 16:20:53.659035 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":125,"thread_start_us":92,"threads_started":1}
I20260809 16:20:53.659262 1613 tablet_service.cc:1467] Processing CreateTablet for tablet 49a3de38697942c3b179fb303f046d80 (DEFAULT_TABLE table=test-table [id=08266703f67241f8974d6f85c486afc8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:20:53.659699 1613 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 49a3de38697942c3b179fb303f046d80. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:20:53.659744 1172 tablet_service.cc:1467] Processing CreateTablet for tablet 49a3de38697942c3b179fb303f046d80 (DEFAULT_TABLE table=test-table [id=08266703f67241f8974d6f85c486afc8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:20:53.660068 1172 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 49a3de38697942c3b179fb303f046d80. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:20:53.663744 1757 tablet_bootstrap.cc:492] T 49a3de38697942c3b179fb303f046d80 P 9525d3dbf73f46fa93cf72ad9cce21f2: Bootstrap starting.
I20260809 16:20:53.664113 1758 tablet_bootstrap.cc:492] T 49a3de38697942c3b179fb303f046d80 P fab00ea5697f40b8a24b8c4fdc8084e0: Bootstrap starting.
I20260809 16:20:53.664346 1380 tablet_service.cc:1467] Processing CreateTablet for tablet 49a3de38697942c3b179fb303f046d80 (DEFAULT_TABLE table=test-table [id=08266703f67241f8974d6f85c486afc8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:20:53.664580 1757 tablet_bootstrap.cc:654] T 49a3de38697942c3b179fb303f046d80 P 9525d3dbf73f46fa93cf72ad9cce21f2: Neither blocks nor log segments found. Creating new log.
I20260809 16:20:53.664608 1380 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 49a3de38697942c3b179fb303f046d80. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:20:53.665020 1758 tablet_bootstrap.cc:654] T 49a3de38697942c3b179fb303f046d80 P fab00ea5697f40b8a24b8c4fdc8084e0: Neither blocks nor log segments found. Creating new log.
I20260809 16:20:53.665040 1757 log.cc:824] T 49a3de38697942c3b179fb303f046d80 P 9525d3dbf73f46fa93cf72ad9cce21f2: Log is configured to *not* fsync() on all Append() calls
I20260809 16:20:53.665380 1758 log.cc:824] T 49a3de38697942c3b179fb303f046d80 P fab00ea5697f40b8a24b8c4fdc8084e0: Log is configured to *not* fsync() on all Append() calls
I20260809 16:20:53.666121 1758 tablet_bootstrap.cc:492] T 49a3de38697942c3b179fb303f046d80 P fab00ea5697f40b8a24b8c4fdc8084e0: No bootstrap required, opened a new log
I20260809 16:20:53.666119 1757 tablet_bootstrap.cc:492] T 49a3de38697942c3b179fb303f046d80 P 9525d3dbf73f46fa93cf72ad9cce21f2: No bootstrap required, opened a new log
I20260809 16:20:53.666196 1758 ts_tablet_manager.cc:1397] T 49a3de38697942c3b179fb303f046d80 P fab00ea5697f40b8a24b8c4fdc8084e0: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 16:20:53.666206 1757 ts_tablet_manager.cc:1397] T 49a3de38697942c3b179fb303f046d80 P 9525d3dbf73f46fa93cf72ad9cce21f2: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260809 16:20:53.667852 1762 tablet_bootstrap.cc:492] T 49a3de38697942c3b179fb303f046d80 P 004c79d477944649ab47d6f916e125fd: Bootstrap starting.
I20260809 16:20:53.668159 1758 raft_consensus.cc:348] T 49a3de38697942c3b179fb303f046d80 P fab00ea5697f40b8a24b8c4fdc8084e0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9525d3dbf73f46fa93cf72ad9cce21f2" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 41441 } } peers { permanent_uuid: "004c79d477944649ab47d6f916e125fd" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 39867 } } peers { permanent_uuid: "fab00ea5697f40b8a24b8c4fdc8084e0" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 33285 } }
I20260809 16:20:53.668159 1757 raft_consensus.cc:348] T 49a3de38697942c3b179fb303f046d80 P 9525d3dbf73f46fa93cf72ad9cce21f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9525d3dbf73f46fa93cf72ad9cce21f2" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 41441 } } peers { permanent_uuid: "004c79d477944649ab47d6f916e125fd" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 39867 } } peers { permanent_uuid: "fab00ea5697f40b8a24b8c4fdc8084e0" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 33285 } }
I20260809 16:20:53.668330 1758 raft_consensus.cc:374] T 49a3de38697942c3b179fb303f046d80 P fab00ea5697f40b8a24b8c4fdc8084e0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:20:53.668330 1757 raft_consensus.cc:374] T 49a3de38697942c3b179fb303f046d80 P 9525d3dbf73f46fa93cf72ad9cce21f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:20:53.668370 1757 raft_consensus.cc:729] T 49a3de38697942c3b179fb303f046d80 P 9525d3dbf73f46fa93cf72ad9cce21f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9525d3dbf73f46fa93cf72ad9cce21f2, State: Initialized, Role: FOLLOWER
I20260809 16:20:53.668370 1758 raft_consensus.cc:729] T 49a3de38697942c3b179fb303f046d80 P fab00ea5697f40b8a24b8c4fdc8084e0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fab00ea5697f40b8a24b8c4fdc8084e0, State: Initialized, Role: FOLLOWER
I20260809 16:20:53.668493 1762 tablet_bootstrap.cc:654] T 49a3de38697942c3b179fb303f046d80 P 004c79d477944649ab47d6f916e125fd: Neither blocks nor log segments found. Creating new log.
I20260809 16:20:53.668488 1757 consensus_queue.cc:260] T 49a3de38697942c3b179fb303f046d80 P 9525d3dbf73f46fa93cf72ad9cce21f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9525d3dbf73f46fa93cf72ad9cce21f2" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 41441 } } peers { permanent_uuid: "004c79d477944649ab47d6f916e125fd" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 39867 } } peers { permanent_uuid: "fab00ea5697f40b8a24b8c4fdc8084e0" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 33285 } }
I20260809 16:20:53.668502 1758 consensus_queue.cc:260] T 49a3de38697942c3b179fb303f046d80 P fab00ea5697f40b8a24b8c4fdc8084e0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9525d3dbf73f46fa93cf72ad9cce21f2" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 41441 } } peers { permanent_uuid: "004c79d477944649ab47d6f916e125fd" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 39867 } } peers { permanent_uuid: "fab00ea5697f40b8a24b8c4fdc8084e0" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 33285 } }
I20260809 16:20:53.668784 1757 ts_tablet_manager.cc:1428] T 49a3de38697942c3b179fb303f046d80 P 9525d3dbf73f46fa93cf72ad9cce21f2: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 16:20:53.668784 1758 ts_tablet_manager.cc:1428] T 49a3de38697942c3b179fb303f046d80 P fab00ea5697f40b8a24b8c4fdc8084e0: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 16:20:53.668844 1707 heartbeater.cc:503] Master 127.21.51.190:37941 was elected leader, sending a full tablet report...
I20260809 16:20:53.668856 1762 log.cc:824] T 49a3de38697942c3b179fb303f046d80 P 004c79d477944649ab47d6f916e125fd: Log is configured to *not* fsync() on all Append() calls
I20260809 16:20:53.668901 1269 heartbeater.cc:503] Master 127.21.51.190:37941 was elected leader, sending a full tablet report...
I20260809 16:20:53.670017 1762 tablet_bootstrap.cc:492] T 49a3de38697942c3b179fb303f046d80 P 004c79d477944649ab47d6f916e125fd: No bootstrap required, opened a new log
I20260809 16:20:53.670071 1762 ts_tablet_manager.cc:1397] T 49a3de38697942c3b179fb303f046d80 P 004c79d477944649ab47d6f916e125fd: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 16:20:53.671440 1762 raft_consensus.cc:348] T 49a3de38697942c3b179fb303f046d80 P 004c79d477944649ab47d6f916e125fd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9525d3dbf73f46fa93cf72ad9cce21f2" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 41441 } } peers { permanent_uuid: "004c79d477944649ab47d6f916e125fd" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 39867 } } peers { permanent_uuid: "fab00ea5697f40b8a24b8c4fdc8084e0" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 33285 } }
I20260809 16:20:53.671558 1762 raft_consensus.cc:374] T 49a3de38697942c3b179fb303f046d80 P 004c79d477944649ab47d6f916e125fd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:20:53.671581 1762 raft_consensus.cc:729] T 49a3de38697942c3b179fb303f046d80 P 004c79d477944649ab47d6f916e125fd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 004c79d477944649ab47d6f916e125fd, State: Initialized, Role: FOLLOWER
I20260809 16:20:53.671661 1762 consensus_queue.cc:260] T 49a3de38697942c3b179fb303f046d80 P 004c79d477944649ab47d6f916e125fd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9525d3dbf73f46fa93cf72ad9cce21f2" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 41441 } } peers { permanent_uuid: "004c79d477944649ab47d6f916e125fd" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 39867 } } peers { permanent_uuid: "fab00ea5697f40b8a24b8c4fdc8084e0" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 33285 } }
I20260809 16:20:53.671850 1762 ts_tablet_manager.cc:1428] T 49a3de38697942c3b179fb303f046d80 P 004c79d477944649ab47d6f916e125fd: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260809 16:20:53.671932 1486 heartbeater.cc:503] Master 127.21.51.190:37941 was elected leader, sending a full tablet report...
W20260809 16:20:53.703837 1487 tablet.cc:2369] T 49a3de38697942c3b179fb303f046d80 P 004c79d477944649ab47d6f916e125fd: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 16:20:53.781082 1270 tablet.cc:2369] T 49a3de38697942c3b179fb303f046d80 P fab00ea5697f40b8a24b8c4fdc8084e0: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 16:20:53.860427 1714 tablet.cc:2369] T 49a3de38697942c3b179fb303f046d80 P 9525d3dbf73f46fa93cf72ad9cce21f2: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 16:20:53.864355 1768 raft_consensus.cc:482] T 49a3de38697942c3b179fb303f046d80 P 004c79d477944649ab47d6f916e125fd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 16:20:53.864485 1768 raft_consensus.cc:504] T 49a3de38697942c3b179fb303f046d80 P 004c79d477944649ab47d6f916e125fd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9525d3dbf73f46fa93cf72ad9cce21f2" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 41441 } } peers { permanent_uuid: "004c79d477944649ab47d6f916e125fd" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 39867 } } peers { permanent_uuid: "fab00ea5697f40b8a24b8c4fdc8084e0" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 33285 } }
I20260809 16:20:53.864934 1768 leader_election.cc:288] T 49a3de38697942c3b179fb303f046d80 P 004c79d477944649ab47d6f916e125fd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9525d3dbf73f46fa93cf72ad9cce21f2 (127.21.51.131:41441), fab00ea5697f40b8a24b8c4fdc8084e0 (127.21.51.129:33285)
I20260809 16:20:53.867900 1756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:53.865199 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.51.130:57047 (local address 127.21.51.129:33285)
0809 16:20:53.865263 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0809 16:20:53.865265 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:20:53.865411 (+ 146us) server_negotiation.cc:407] Connection header received
0809 16:20:53.865463 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:20:53.865464 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:20:53.865473 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:20:53.865543 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:20:53.865689 (+ 146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:53.866142 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:53.867085 (+ 943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:53.867450 (+ 365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:53.867532 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:53.867541 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0809 16:20:53.867796 (+ 255us) server_negotiation.cc:299] Negotiation successful
0809 16:20:53.867833 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260809 16:20:53.867910 1755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:53.865205 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.51.130:53003 (local address 127.21.51.131:41441)
0809 16:20:53.865267 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0809 16:20:53.865269 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:20:53.865306 (+ 37us) server_negotiation.cc:407] Connection header received
0809 16:20:53.865340 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:20:53.865341 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:20:53.865352 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:20:53.865408 (+ 56us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:20:53.865595 (+ 187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:53.866120 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:53.867085 (+ 965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:53.867450 (+ 365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:53.867634 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:53.867646 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0809 16:20:53.867802 (+ 156us) server_negotiation.cc:299] Negotiation successful
0809 16:20:53.867841 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260809 16:20:53.867924 1770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:53.865153 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.129:33285 (local address 127.21.51.130:57047)
0809 16:20:53.865328 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0809 16:20:53.865339 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 16:20:53.865400 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:20:53.865544 (+ 144us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:20:53.865545 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:20:53.865546 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:20:53.865630 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:20:53.865632 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:20:53.866239 (+ 607us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:20:53.866240 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:20:53.866994 (+ 754us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:20:53.866996 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:20:53.867592 (+ 596us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:20:53.867594 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:20:53.867685 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:53.867689 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 16:20:53.867773 (+ 84us) client_negotiation.cc:240] Negotiation successful
0809 16:20:53.867817 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":159,"mutex_wait_us":3,"thread_start_us":104,"threads_started":1}
I20260809 16:20:53.867933 1318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:53.865141 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.131:41441 (local address 127.21.51.130:53003)
0809 16:20:53.865199 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0809 16:20:53.865211 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 16:20:53.865260 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:20:53.865432 (+ 172us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:20:53.865433 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:20:53.865435 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:20:53.865516 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:20:53.865518 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:20:53.866204 (+ 686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:20:53.866206 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:20:53.866989 (+ 783us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:20:53.866991 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:20:53.867634 (+ 643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:20:53.867635 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:20:53.867705 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:53.867707 (+ 2us) client_negotiation.cc:769] Sending connection context
0809 16:20:53.867775 (+ 68us) client_negotiation.cc:240] Negotiation successful
0809 16:20:53.867817 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38,"mutex_wait_us":1}
I20260809 16:20:53.868269 1645 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "49a3de38697942c3b179fb303f046d80" candidate_uuid: "004c79d477944649ab47d6f916e125fd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9525d3dbf73f46fa93cf72ad9cce21f2" is_pre_election: true
I20260809 16:20:53.868265 1202 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "49a3de38697942c3b179fb303f046d80" candidate_uuid: "004c79d477944649ab47d6f916e125fd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fab00ea5697f40b8a24b8c4fdc8084e0" is_pre_election: true
I20260809 16:20:53.868431 1645 raft_consensus.cc:2454] T 49a3de38697942c3b179fb303f046d80 P 9525d3dbf73f46fa93cf72ad9cce21f2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 004c79d477944649ab47d6f916e125fd in term 0.
I20260809 16:20:53.868551 1202 raft_consensus.cc:2454] T 49a3de38697942c3b179fb303f046d80 P fab00ea5697f40b8a24b8c4fdc8084e0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 004c79d477944649ab47d6f916e125fd in term 0.
I20260809 16:20:53.868752 1310 leader_election.cc:302] T 49a3de38697942c3b179fb303f046d80 P 004c79d477944649ab47d6f916e125fd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 004c79d477944649ab47d6f916e125fd, 9525d3dbf73f46fa93cf72ad9cce21f2; no voters:
I20260809 16:20:53.868894 1768 raft_consensus.cc:2781] T 49a3de38697942c3b179fb303f046d80 P 004c79d477944649ab47d6f916e125fd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 16:20:53.868957 1768 raft_consensus.cc:482] T 49a3de38697942c3b179fb303f046d80 P 004c79d477944649ab47d6f916e125fd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 16:20:53.868991 1768 raft_consensus.cc:3037] T 49a3de38697942c3b179fb303f046d80 P 004c79d477944649ab47d6f916e125fd [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:20:53.869985 1768 raft_consensus.cc:504] T 49a3de38697942c3b179fb303f046d80 P 004c79d477944649ab47d6f916e125fd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9525d3dbf73f46fa93cf72ad9cce21f2" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 41441 } } peers { permanent_uuid: "004c79d477944649ab47d6f916e125fd" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 39867 } } peers { permanent_uuid: "fab00ea5697f40b8a24b8c4fdc8084e0" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 33285 } }
I20260809 16:20:53.870143 1768 leader_election.cc:288] T 49a3de38697942c3b179fb303f046d80 P 004c79d477944649ab47d6f916e125fd [CANDIDATE]: Term 1 election: Requested vote from peers 9525d3dbf73f46fa93cf72ad9cce21f2 (127.21.51.131:41441), fab00ea5697f40b8a24b8c4fdc8084e0 (127.21.51.129:33285)
I20260809 16:20:53.870285 1645 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "49a3de38697942c3b179fb303f046d80" candidate_uuid: "004c79d477944649ab47d6f916e125fd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9525d3dbf73f46fa93cf72ad9cce21f2"
I20260809 16:20:53.870363 1645 raft_consensus.cc:3037] T 49a3de38697942c3b179fb303f046d80 P 9525d3dbf73f46fa93cf72ad9cce21f2 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:20:53.870337 1202 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "49a3de38697942c3b179fb303f046d80" candidate_uuid: "004c79d477944649ab47d6f916e125fd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fab00ea5697f40b8a24b8c4fdc8084e0"
I20260809 16:20:53.870462 1202 raft_consensus.cc:3037] T 49a3de38697942c3b179fb303f046d80 P fab00ea5697f40b8a24b8c4fdc8084e0 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:20:53.870998 1645 raft_consensus.cc:2454] T 49a3de38697942c3b179fb303f046d80 P 9525d3dbf73f46fa93cf72ad9cce21f2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 004c79d477944649ab47d6f916e125fd in term 1.
I20260809 16:20:53.871157 1310 leader_election.cc:302] T 49a3de38697942c3b179fb303f046d80 P 004c79d477944649ab47d6f916e125fd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 004c79d477944649ab47d6f916e125fd, 9525d3dbf73f46fa93cf72ad9cce21f2; no voters:
I20260809 16:20:53.871255 1768 raft_consensus.cc:2781] T 49a3de38697942c3b179fb303f046d80 P 004c79d477944649ab47d6f916e125fd [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:20:53.871358 1202 raft_consensus.cc:2454] T 49a3de38697942c3b179fb303f046d80 P fab00ea5697f40b8a24b8c4fdc8084e0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 004c79d477944649ab47d6f916e125fd in term 1.
I20260809 16:20:53.871407 1768 raft_consensus.cc:686] T 49a3de38697942c3b179fb303f046d80 P 004c79d477944649ab47d6f916e125fd [term 1 LEADER]: Becoming Leader. State: Replica: 004c79d477944649ab47d6f916e125fd, State: Running, Role: LEADER
I20260809 16:20:53.871683 1768 consensus_queue.cc:237] T 49a3de38697942c3b179fb303f046d80 P 004c79d477944649ab47d6f916e125fd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9525d3dbf73f46fa93cf72ad9cce21f2" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 41441 } } peers { permanent_uuid: "004c79d477944649ab47d6f916e125fd" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 39867 } } peers { permanent_uuid: "fab00ea5697f40b8a24b8c4fdc8084e0" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 33285 } }
I20260809 16:20:53.872541 951 catalog_manager.cc:5495] T 49a3de38697942c3b179fb303f046d80 P 004c79d477944649ab47d6f916e125fd reported cstate change: term changed from 0 to 1, leader changed from <none> to 004c79d477944649ab47d6f916e125fd (127.21.51.130). New cstate: current_term: 1 leader_uuid: "004c79d477944649ab47d6f916e125fd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9525d3dbf73f46fa93cf72ad9cce21f2" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 41441 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "004c79d477944649ab47d6f916e125fd" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 39867 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fab00ea5697f40b8a24b8c4fdc8084e0" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 33285 } health_report { overall_health: UNKNOWN } } }
I20260809 16:20:53.882648 1754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:53.880326 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47052 (local address 127.21.51.130:39867)
0809 16:20:53.880369 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0809 16:20:53.880371 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:20:53.880382 (+ 11us) server_negotiation.cc:407] Connection header received
0809 16:20:53.880452 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:20:53.880453 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:20:53.880469 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:20:53.880532 (+ 63us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 16:20:53.880676 (+ 144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:53.881146 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:53.881888 (+ 742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:53.882098 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:53.882167 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:53.882328 (+ 161us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 16:20:53.882425 (+ 97us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 16:20:53.882503 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0809 16:20:53.882556 (+ 53us) server_negotiation.cc:299] Negotiation successful
0809 16:20:53.882575 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260809 16:20:53.884465 1202 raft_consensus.cc:1261] T 49a3de38697942c3b179fb303f046d80 P fab00ea5697f40b8a24b8c4fdc8084e0 [term 1 FOLLOWER]: Refusing update from remote peer 004c79d477944649ab47d6f916e125fd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 16:20:53.884513 1645 raft_consensus.cc:1261] T 49a3de38697942c3b179fb303f046d80 P 9525d3dbf73f46fa93cf72ad9cce21f2 [term 1 FOLLOWER]: Refusing update from remote peer 004c79d477944649ab47d6f916e125fd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 16:20:53.884878 1771 consensus_queue.cc:1035] T 49a3de38697942c3b179fb303f046d80 P 004c79d477944649ab47d6f916e125fd [LEADER]: Connected to new peer: Peer: permanent_uuid: "9525d3dbf73f46fa93cf72ad9cce21f2" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 41441 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 16:20:53.885000 1768 consensus_queue.cc:1035] T 49a3de38697942c3b179fb303f046d80 P 004c79d477944649ab47d6f916e125fd [LEADER]: Connected to new peer: Peer: permanent_uuid: "fab00ea5697f40b8a24b8c4fdc8084e0" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 33285 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 16:20:53.887151 1778 mvcc.cc:205] Tried to move back new op lower bound from 7316653891107999744 to 7316653891059015680. Current Snapshot: MvccSnapshot[applied={T|T < 7316653891107999744}]
I20260809 16:20:53.887364 1788 mvcc.cc:205] Tried to move back new op lower bound from 7316653891107999744 to 7316653891059015680. Current Snapshot: MvccSnapshot[applied={T|T < 7316653891107999744}]
I20260809 16:20:53.889860 955 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43632:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 16:20:53.889951 955 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43632:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 16:20:53.890695 955 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 40f8427e400f4dab9038e3273f59f1d0: Sending DeleteTablet for 3 replicas of tablet 49a3de38697942c3b179fb303f046d80
I20260809 16:20:53.891033 1613 tablet_service.cc:1514] Processing DeleteTablet for tablet 49a3de38697942c3b179fb303f046d80 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:20:53 UTC) from {username='oryx', principal='oryx/127.21.51.190@KRBTEST.COM'} at 127.0.0.1:37256
I20260809 16:20:53.891070 1172 tablet_service.cc:1514] Processing DeleteTablet for tablet 49a3de38697942c3b179fb303f046d80 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:20:53 UTC) from {username='oryx', principal='oryx/127.21.51.190@KRBTEST.COM'} at 127.0.0.1:39410
I20260809 16:20:53.891247 1380 tablet_service.cc:1514] Processing DeleteTablet for tablet 49a3de38697942c3b179fb303f046d80 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:20:53 UTC) from {username='oryx', principal='oryx/127.21.51.190@KRBTEST.COM'} at 127.0.0.1:47044
I20260809 16:20:53.891283 955 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43632:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 16:20:53.891438 955 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 16:20:53.891464 1802 tablet_replica.cc:330] T 49a3de38697942c3b179fb303f046d80 P 004c79d477944649ab47d6f916e125fd: stopping tablet replica
I20260809 16:20:53.891553 1802 raft_consensus.cc:2229] T 49a3de38697942c3b179fb303f046d80 P 004c79d477944649ab47d6f916e125fd [term 1 LEADER]: Raft consensus shutting down.
I20260809 16:20:53.891676 1802 raft_consensus.cc:2258] T 49a3de38697942c3b179fb303f046d80 P 004c79d477944649ab47d6f916e125fd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 16:20:53.891820 1801 tablet_replica.cc:330] T 49a3de38697942c3b179fb303f046d80 P fab00ea5697f40b8a24b8c4fdc8084e0: stopping tablet replica
I20260809 16:20:53.891913 1801 raft_consensus.cc:2229] T 49a3de38697942c3b179fb303f046d80 P fab00ea5697f40b8a24b8c4fdc8084e0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 16:20:53.891997 1801 raft_consensus.cc:2258] T 49a3de38697942c3b179fb303f046d80 P fab00ea5697f40b8a24b8c4fdc8084e0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 16:20:53.892462 1801 ts_tablet_manager.cc:1905] T 49a3de38697942c3b179fb303f046d80 P fab00ea5697f40b8a24b8c4fdc8084e0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 16:20:53.892802 1172 tablet_service.cc:1467] Processing CreateTablet for tablet 71c481afdbf944d9bca6c12687fd0203 (DEFAULT_TABLE table=test-table [id=aada97400ce84b4e9e5af872bbb0931d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:20:53.892859 1613 tablet_service.cc:1467] Processing CreateTablet for tablet 71c481afdbf944d9bca6c12687fd0203 (DEFAULT_TABLE table=test-table [id=aada97400ce84b4e9e5af872bbb0931d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:20:53.892923 1172 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 71c481afdbf944d9bca6c12687fd0203. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:20:53.892983 1613 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 71c481afdbf944d9bca6c12687fd0203. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:20:53.894248 1758 tablet_bootstrap.cc:492] T 71c481afdbf944d9bca6c12687fd0203 P fab00ea5697f40b8a24b8c4fdc8084e0: Bootstrap starting.
I20260809 16:20:53.894304 1801 ts_tablet_manager.cc:1918] T 49a3de38697942c3b179fb303f046d80 P fab00ea5697f40b8a24b8c4fdc8084e0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 16:20:53.894385 1801 log.cc:1196] T 49a3de38697942c3b179fb303f046d80 P fab00ea5697f40b8a24b8c4fdc8084e0: Deleting WAL directory at /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786292417722232-21710-0/minicluster-data/ts-0/wal/wals/49a3de38697942c3b179fb303f046d80
I20260809 16:20:53.894409 1757 tablet_bootstrap.cc:492] T 71c481afdbf944d9bca6c12687fd0203 P 9525d3dbf73f46fa93cf72ad9cce21f2: Bootstrap starting.
I20260809 16:20:53.894691 1758 tablet_bootstrap.cc:654] T 71c481afdbf944d9bca6c12687fd0203 P fab00ea5697f40b8a24b8c4fdc8084e0: Neither blocks nor log segments found. Creating new log.
I20260809 16:20:53.894721 1801 ts_tablet_manager.cc:1939] T 49a3de38697942c3b179fb303f046d80 P fab00ea5697f40b8a24b8c4fdc8084e0: Deleting consensus metadata
I20260809 16:20:53.894734 1380 tablet_service.cc:1467] Processing CreateTablet for tablet 71c481afdbf944d9bca6c12687fd0203 (DEFAULT_TABLE table=test-table [id=aada97400ce84b4e9e5af872bbb0931d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:20:53.894831 1380 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 71c481afdbf944d9bca6c12687fd0203. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:20:53.894866 1757 tablet_bootstrap.cc:654] T 71c481afdbf944d9bca6c12687fd0203 P 9525d3dbf73f46fa93cf72ad9cce21f2: Neither blocks nor log segments found. Creating new log.
I20260809 16:20:53.895499 1757 tablet_bootstrap.cc:492] T 71c481afdbf944d9bca6c12687fd0203 P 9525d3dbf73f46fa93cf72ad9cce21f2: No bootstrap required, opened a new log
I20260809 16:20:53.895557 1757 ts_tablet_manager.cc:1397] T 71c481afdbf944d9bca6c12687fd0203 P 9525d3dbf73f46fa93cf72ad9cce21f2: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260809 16:20:53.895725 1757 raft_consensus.cc:348] T 71c481afdbf944d9bca6c12687fd0203 P 9525d3dbf73f46fa93cf72ad9cce21f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fab00ea5697f40b8a24b8c4fdc8084e0" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 33285 } } peers { permanent_uuid: "004c79d477944649ab47d6f916e125fd" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 39867 } } peers { permanent_uuid: "9525d3dbf73f46fa93cf72ad9cce21f2" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 41441 } }
I20260809 16:20:53.895802 1757 raft_consensus.cc:374] T 71c481afdbf944d9bca6c12687fd0203 P 9525d3dbf73f46fa93cf72ad9cce21f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:20:53.895823 1757 raft_consensus.cc:729] T 71c481afdbf944d9bca6c12687fd0203 P 9525d3dbf73f46fa93cf72ad9cce21f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9525d3dbf73f46fa93cf72ad9cce21f2, State: Initialized, Role: FOLLOWER
I20260809 16:20:53.895869 1757 consensus_queue.cc:260] T 71c481afdbf944d9bca6c12687fd0203 P 9525d3dbf73f46fa93cf72ad9cce21f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fab00ea5697f40b8a24b8c4fdc8084e0" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 33285 } } peers { permanent_uuid: "004c79d477944649ab47d6f916e125fd" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 39867 } } peers { permanent_uuid: "9525d3dbf73f46fa93cf72ad9cce21f2" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 41441 } }
I20260809 16:20:53.895955 1757 ts_tablet_manager.cc:1428] T 71c481afdbf944d9bca6c12687fd0203 P 9525d3dbf73f46fa93cf72ad9cce21f2: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260809 16:20:53.896255 1762 tablet_bootstrap.cc:492] T 71c481afdbf944d9bca6c12687fd0203 P 004c79d477944649ab47d6f916e125fd: Bootstrap starting.
I20260809 16:20:53.896314 934 catalog_manager.cc:4841] TS fab00ea5697f40b8a24b8c4fdc8084e0 (127.21.51.129:33285): tablet 49a3de38697942c3b179fb303f046d80 (table test-table [id=08266703f67241f8974d6f85c486afc8]) successfully deleted
I20260809 16:20:53.896347 1802 ts_tablet_manager.cc:1905] T 49a3de38697942c3b179fb303f046d80 P 004c79d477944649ab47d6f916e125fd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 16:20:53.896664 1799 tablet_replica.cc:330] T 49a3de38697942c3b179fb303f046d80 P 9525d3dbf73f46fa93cf72ad9cce21f2: stopping tablet replica
I20260809 16:20:53.896677 1762 tablet_bootstrap.cc:654] T 71c481afdbf944d9bca6c12687fd0203 P 004c79d477944649ab47d6f916e125fd: Neither blocks nor log segments found. Creating new log.
I20260809 16:20:53.897176 1799 raft_consensus.cc:2229] T 49a3de38697942c3b179fb303f046d80 P 9525d3dbf73f46fa93cf72ad9cce21f2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 16:20:53.897292 1799 raft_consensus.cc:2258] T 49a3de38697942c3b179fb303f046d80 P 9525d3dbf73f46fa93cf72ad9cce21f2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 16:20:53.897699 1799 ts_tablet_manager.cc:1905] T 49a3de38697942c3b179fb303f046d80 P 9525d3dbf73f46fa93cf72ad9cce21f2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 16:20:53.897961 1802 ts_tablet_manager.cc:1918] T 49a3de38697942c3b179fb303f046d80 P 004c79d477944649ab47d6f916e125fd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 16:20:53.898054 1802 log.cc:1196] T 49a3de38697942c3b179fb303f046d80 P 004c79d477944649ab47d6f916e125fd: Deleting WAL directory at /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786292417722232-21710-0/minicluster-data/ts-1/wal/wals/49a3de38697942c3b179fb303f046d80
I20260809 16:20:53.898324 1802 ts_tablet_manager.cc:1939] T 49a3de38697942c3b179fb303f046d80 P 004c79d477944649ab47d6f916e125fd: Deleting consensus metadata
I20260809 16:20:53.898725 935 catalog_manager.cc:4841] TS 004c79d477944649ab47d6f916e125fd (127.21.51.130:39867): tablet 49a3de38697942c3b179fb303f046d80 (table test-table [id=08266703f67241f8974d6f85c486afc8]) successfully deleted
I20260809 16:20:53.899326 1799 ts_tablet_manager.cc:1918] T 49a3de38697942c3b179fb303f046d80 P 9525d3dbf73f46fa93cf72ad9cce21f2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 16:20:53.899389 1799 log.cc:1196] T 49a3de38697942c3b179fb303f046d80 P 9525d3dbf73f46fa93cf72ad9cce21f2: Deleting WAL directory at /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786292417722232-21710-0/minicluster-data/ts-2/wal/wals/49a3de38697942c3b179fb303f046d80
I20260809 16:20:53.899623 1799 ts_tablet_manager.cc:1939] T 49a3de38697942c3b179fb303f046d80 P 9525d3dbf73f46fa93cf72ad9cce21f2: Deleting consensus metadata
I20260809 16:20:53.899994 936 catalog_manager.cc:4841] TS 9525d3dbf73f46fa93cf72ad9cce21f2 (127.21.51.131:41441): tablet 49a3de38697942c3b179fb303f046d80 (table test-table [id=08266703f67241f8974d6f85c486afc8]) successfully deleted
I20260809 16:20:53.900199 1762 tablet_bootstrap.cc:492] T 71c481afdbf944d9bca6c12687fd0203 P 004c79d477944649ab47d6f916e125fd: No bootstrap required, opened a new log
I20260809 16:20:53.900242 1762 ts_tablet_manager.cc:1397] T 71c481afdbf944d9bca6c12687fd0203 P 004c79d477944649ab47d6f916e125fd: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260809 16:20:53.900405 1762 raft_consensus.cc:348] T 71c481afdbf944d9bca6c12687fd0203 P 004c79d477944649ab47d6f916e125fd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fab00ea5697f40b8a24b8c4fdc8084e0" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 33285 } } peers { permanent_uuid: "004c79d477944649ab47d6f916e125fd" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 39867 } } peers { permanent_uuid: "9525d3dbf73f46fa93cf72ad9cce21f2" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 41441 } }
I20260809 16:20:53.900466 1762 raft_consensus.cc:374] T 71c481afdbf944d9bca6c12687fd0203 P 004c79d477944649ab47d6f916e125fd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:20:53.900486 1762 raft_consensus.cc:729] T 71c481afdbf944d9bca6c12687fd0203 P 004c79d477944649ab47d6f916e125fd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 004c79d477944649ab47d6f916e125fd, State: Initialized, Role: FOLLOWER
I20260809 16:20:53.900537 1762 consensus_queue.cc:260] T 71c481afdbf944d9bca6c12687fd0203 P 004c79d477944649ab47d6f916e125fd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fab00ea5697f40b8a24b8c4fdc8084e0" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 33285 } } peers { permanent_uuid: "004c79d477944649ab47d6f916e125fd" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 39867 } } peers { permanent_uuid: "9525d3dbf73f46fa93cf72ad9cce21f2" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 41441 } }
I20260809 16:20:53.900638 1762 ts_tablet_manager.cc:1428] T 71c481afdbf944d9bca6c12687fd0203 P 004c79d477944649ab47d6f916e125fd: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260809 16:20:53.901060 1758 tablet_bootstrap.cc:492] T 71c481afdbf944d9bca6c12687fd0203 P fab00ea5697f40b8a24b8c4fdc8084e0: No bootstrap required, opened a new log
I20260809 16:20:53.901121 1758 ts_tablet_manager.cc:1397] T 71c481afdbf944d9bca6c12687fd0203 P fab00ea5697f40b8a24b8c4fdc8084e0: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.000s
I20260809 16:20:53.901510 1758 raft_consensus.cc:348] T 71c481afdbf944d9bca6c12687fd0203 P fab00ea5697f40b8a24b8c4fdc8084e0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fab00ea5697f40b8a24b8c4fdc8084e0" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 33285 } } peers { permanent_uuid: "004c79d477944649ab47d6f916e125fd" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 39867 } } peers { permanent_uuid: "9525d3dbf73f46fa93cf72ad9cce21f2" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 41441 } }
I20260809 16:20:53.901587 1758 raft_consensus.cc:374] T 71c481afdbf944d9bca6c12687fd0203 P fab00ea5697f40b8a24b8c4fdc8084e0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:20:53.901603 1758 raft_consensus.cc:729] T 71c481afdbf944d9bca6c12687fd0203 P fab00ea5697f40b8a24b8c4fdc8084e0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fab00ea5697f40b8a24b8c4fdc8084e0, State: Initialized, Role: FOLLOWER
I20260809 16:20:53.901633 1758 consensus_queue.cc:260] T 71c481afdbf944d9bca6c12687fd0203 P fab00ea5697f40b8a24b8c4fdc8084e0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fab00ea5697f40b8a24b8c4fdc8084e0" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 33285 } } peers { permanent_uuid: "004c79d477944649ab47d6f916e125fd" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 39867 } } peers { permanent_uuid: "9525d3dbf73f46fa93cf72ad9cce21f2" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 41441 } }
I20260809 16:20:53.901723 1758 ts_tablet_manager.cc:1428] T 71c481afdbf944d9bca6c12687fd0203 P fab00ea5697f40b8a24b8c4fdc8084e0: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260809 16:20:53.982048 1764 raft_consensus.cc:482] T 71c481afdbf944d9bca6c12687fd0203 P fab00ea5697f40b8a24b8c4fdc8084e0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 16:20:53.982142 1764 raft_consensus.cc:504] T 71c481afdbf944d9bca6c12687fd0203 P fab00ea5697f40b8a24b8c4fdc8084e0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fab00ea5697f40b8a24b8c4fdc8084e0" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 33285 } } peers { permanent_uuid: "004c79d477944649ab47d6f916e125fd" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 39867 } } peers { permanent_uuid: "9525d3dbf73f46fa93cf72ad9cce21f2" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 41441 } }
I20260809 16:20:53.982486 1764 leader_election.cc:288] T 71c481afdbf944d9bca6c12687fd0203 P fab00ea5697f40b8a24b8c4fdc8084e0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 004c79d477944649ab47d6f916e125fd (127.21.51.130:39867), 9525d3dbf73f46fa93cf72ad9cce21f2 (127.21.51.131:41441)
I20260809 16:20:53.986213 1755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:53.982843 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.51.129:55403 (local address 127.21.51.131:41441)
0809 16:20:53.982925 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0809 16:20:53.982927 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:20:53.983068 (+ 141us) server_negotiation.cc:407] Connection header received
0809 16:20:53.983203 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:20:53.983205 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:20:53.983217 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:20:53.983273 (+ 56us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:20:53.983434 (+ 161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:53.984129 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:53.985313 (+ 1184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:53.985816 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:53.985892 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:53.985901 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0809 16:20:53.986098 (+ 197us) server_negotiation.cc:299] Negotiation successful
0809 16:20:53.986140 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260809 16:20:53.986444 1754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:53.982910 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.51.129:42737 (local address 127.21.51.130:39867)
0809 16:20:53.982955 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0809 16:20:53.982957 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:20:53.982966 (+ 9us) server_negotiation.cc:407] Connection header received
0809 16:20:53.983002 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:20:53.983002 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:20:53.983017 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:20:53.983082 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:20:53.983296 (+ 214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:53.984050 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:53.985532 (+ 1482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:53.986040 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:53.986123 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:53.986131 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0809 16:20:53.986350 (+ 219us) server_negotiation.cc:299] Negotiation successful
0809 16:20:53.986393 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260809 16:20:53.986488 1807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:53.982679 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.131:41441 (local address 127.21.51.129:55403)
0809 16:20:53.983001 (+ 322us) negotiation.cc:107] Waiting for socket to connect
0809 16:20:53.983013 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 16:20:53.983097 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:20:53.983279 (+ 182us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:20:53.983280 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:20:53.983283 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:20:53.983359 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:20:53.983361 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:20:53.984201 (+ 840us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:20:53.984203 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:20:53.985228 (+ 1025us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:20:53.985231 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:20:53.985928 (+ 697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:20:53.985929 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:20:53.986007 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:53.986011 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 16:20:53.986089 (+ 78us) client_negotiation.cc:240] Negotiation successful
0809 16:20:53.986146 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":307,"mutex_wait_us":37,"thread_start_us":108,"threads_started":1}
I20260809 16:20:53.986681 1809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:53.982675 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.130:39867 (local address 127.21.51.129:42737)
0809 16:20:53.982840 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0809 16:20:53.982853 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 16:20:53.982916 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:20:53.983089 (+ 173us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:20:53.983090 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:20:53.983092 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:20:53.983186 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:20:53.983188 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:20:53.984138 (+ 950us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:20:53.984140 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:20:53.985439 (+ 1299us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:20:53.985442 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:20:53.986167 (+ 725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:20:53.986168 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:20:53.986254 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:53.986259 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 16:20:53.986537 (+ 278us) client_negotiation.cc:240] Negotiation successful
0809 16:20:53.986591 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"mutex_wait_us":19,"thread_start_us":77,"threads_started":1}
I20260809 16:20:53.986749 1645 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "71c481afdbf944d9bca6c12687fd0203" candidate_uuid: "fab00ea5697f40b8a24b8c4fdc8084e0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9525d3dbf73f46fa93cf72ad9cce21f2" is_pre_election: true
I20260809 16:20:53.986850 1645 raft_consensus.cc:2454] T 71c481afdbf944d9bca6c12687fd0203 P 9525d3dbf73f46fa93cf72ad9cce21f2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fab00ea5697f40b8a24b8c4fdc8084e0 in term 0.
I20260809 16:20:53.987097 1412 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "71c481afdbf944d9bca6c12687fd0203" candidate_uuid: "fab00ea5697f40b8a24b8c4fdc8084e0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "004c79d477944649ab47d6f916e125fd" is_pre_election: true
I20260809 16:20:53.987217 1412 raft_consensus.cc:2454] T 71c481afdbf944d9bca6c12687fd0203 P 004c79d477944649ab47d6f916e125fd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fab00ea5697f40b8a24b8c4fdc8084e0 in term 0.
I20260809 16:20:53.987406 1089 leader_election.cc:302] T 71c481afdbf944d9bca6c12687fd0203 P fab00ea5697f40b8a24b8c4fdc8084e0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9525d3dbf73f46fa93cf72ad9cce21f2, fab00ea5697f40b8a24b8c4fdc8084e0; no voters:
I20260809 16:20:53.987615 1764 raft_consensus.cc:2781] T 71c481afdbf944d9bca6c12687fd0203 P fab00ea5697f40b8a24b8c4fdc8084e0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 16:20:53.987654 1764 raft_consensus.cc:482] T 71c481afdbf944d9bca6c12687fd0203 P fab00ea5697f40b8a24b8c4fdc8084e0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 16:20:53.987669 1764 raft_consensus.cc:3037] T 71c481afdbf944d9bca6c12687fd0203 P fab00ea5697f40b8a24b8c4fdc8084e0 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:20:53.988340 1764 raft_consensus.cc:504] T 71c481afdbf944d9bca6c12687fd0203 P fab00ea5697f40b8a24b8c4fdc8084e0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fab00ea5697f40b8a24b8c4fdc8084e0" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 33285 } } peers { permanent_uuid: "004c79d477944649ab47d6f916e125fd" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 39867 } } peers { permanent_uuid: "9525d3dbf73f46fa93cf72ad9cce21f2" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 41441 } }
I20260809 16:20:53.988468 1764 leader_election.cc:288] T 71c481afdbf944d9bca6c12687fd0203 P fab00ea5697f40b8a24b8c4fdc8084e0 [CANDIDATE]: Term 1 election: Requested vote from peers 004c79d477944649ab47d6f916e125fd (127.21.51.130:39867), 9525d3dbf73f46fa93cf72ad9cce21f2 (127.21.51.131:41441)
I20260809 16:20:53.988624 1645 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "71c481afdbf944d9bca6c12687fd0203" candidate_uuid: "fab00ea5697f40b8a24b8c4fdc8084e0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9525d3dbf73f46fa93cf72ad9cce21f2"
I20260809 16:20:53.988741 1645 raft_consensus.cc:3037] T 71c481afdbf944d9bca6c12687fd0203 P 9525d3dbf73f46fa93cf72ad9cce21f2 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:20:53.988726 1412 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "71c481afdbf944d9bca6c12687fd0203" candidate_uuid: "fab00ea5697f40b8a24b8c4fdc8084e0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "004c79d477944649ab47d6f916e125fd"
I20260809 16:20:53.988787 1412 raft_consensus.cc:3037] T 71c481afdbf944d9bca6c12687fd0203 P 004c79d477944649ab47d6f916e125fd [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:20:53.989544 1412 raft_consensus.cc:2454] T 71c481afdbf944d9bca6c12687fd0203 P 004c79d477944649ab47d6f916e125fd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fab00ea5697f40b8a24b8c4fdc8084e0 in term 1.
I20260809 16:20:53.989544 1645 raft_consensus.cc:2454] T 71c481afdbf944d9bca6c12687fd0203 P 9525d3dbf73f46fa93cf72ad9cce21f2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fab00ea5697f40b8a24b8c4fdc8084e0 in term 1.
I20260809 16:20:53.989727 1089 leader_election.cc:302] T 71c481afdbf944d9bca6c12687fd0203 P fab00ea5697f40b8a24b8c4fdc8084e0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9525d3dbf73f46fa93cf72ad9cce21f2, fab00ea5697f40b8a24b8c4fdc8084e0; no voters:
I20260809 16:20:53.989804 1764 raft_consensus.cc:2781] T 71c481afdbf944d9bca6c12687fd0203 P fab00ea5697f40b8a24b8c4fdc8084e0 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:20:53.989945 1764 raft_consensus.cc:686] T 71c481afdbf944d9bca6c12687fd0203 P fab00ea5697f40b8a24b8c4fdc8084e0 [term 1 LEADER]: Becoming Leader. State: Replica: fab00ea5697f40b8a24b8c4fdc8084e0, State: Running, Role: LEADER
I20260809 16:20:53.990015 1764 consensus_queue.cc:237] T 71c481afdbf944d9bca6c12687fd0203 P fab00ea5697f40b8a24b8c4fdc8084e0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fab00ea5697f40b8a24b8c4fdc8084e0" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 33285 } } peers { permanent_uuid: "004c79d477944649ab47d6f916e125fd" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 39867 } } peers { permanent_uuid: "9525d3dbf73f46fa93cf72ad9cce21f2" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 41441 } }
I20260809 16:20:53.990732 954 catalog_manager.cc:5495] T 71c481afdbf944d9bca6c12687fd0203 P fab00ea5697f40b8a24b8c4fdc8084e0 reported cstate change: term changed from 0 to 1, leader changed from <none> to fab00ea5697f40b8a24b8c4fdc8084e0 (127.21.51.129). New cstate: current_term: 1 leader_uuid: "fab00ea5697f40b8a24b8c4fdc8084e0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fab00ea5697f40b8a24b8c4fdc8084e0" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 33285 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "004c79d477944649ab47d6f916e125fd" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 39867 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9525d3dbf73f46fa93cf72ad9cce21f2" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 41441 } health_report { overall_health: UNKNOWN } } }
Aug 09 16:20:54 dist-test-slave-s11t krb5kdc[891](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292453, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.51.190@KRBTEST.COM for oryx/127.21.51.190@KRBTEST.COM
I20260809 16:20:54.040488 1825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:54.033521 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:37941 (local address 127.0.0.1:43636)
0809 16:20:54.033718 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0809 16:20:54.033728 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 16:20:54.033820 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:20:54.034031 (+ 211us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:20:54.034032 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:20:54.034358 (+ 326us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:20:54.034411 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:20:54.034413 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:20:54.035088 (+ 675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:20:54.035090 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:20:54.035620 (+ 530us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:20:54.035621 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:20:54.035958 (+ 337us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:20:54.035959 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:20:54.036014 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:54.036581 (+ 567us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:20:54.036586 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:20:54.038053 (+ 1467us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:20:54.039463 (+ 1410us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:20:54.039469 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:20:54.039471 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:20:54.039673 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:20:54.039886 (+ 213us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:20:54.039887 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:20:54.039887 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:20:54.039955 (+ 68us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:20:54.040179 (+ 224us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:20:54.040185 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:20:54.040242 (+ 57us) client_negotiation.cc:769] Sending connection context
0809 16:20:54.040295 (+ 53us) client_negotiation.cc:240] Negotiation successful
0809 16:20:54.040395 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":93,"threads_started":1}
I20260809 16:20:54.040560 1034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:54.033591 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43636 (local address 127.21.51.190:37941)
0809 16:20:54.033675 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0809 16:20:54.033678 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 16:20:54.033799 (+ 121us) server_negotiation.cc:407] Connection header received
0809 16:20:54.033933 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:20:54.033934 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:20:54.033946 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:20:54.034008 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:20:54.034498 (+ 490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:54.035000 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:54.035703 (+ 703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:54.035888 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:54.035960 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:54.038138 (+ 2178us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:20:54.038139 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:20:54.038140 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:20:54.038146 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:20:54.039389 (+ 1243us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:20:54.039753 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:20:54.039754 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:20:54.039754 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:20:54.039807 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:20:54.040018 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:20:54.040019 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:20:54.040019 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:20:54.040124 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:20:54.040176 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0809 16:20:54.040329 (+ 153us) server_negotiation.cc:299] Negotiation successful
0809 16:20:54.040445 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":38,"server-negotiator.queue_time_us":54}
I20260809 16:20:54.042398 954 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.21.51.190@KRBTEST.COM'} at 127.0.0.1:43636:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260809 16:20:54.042631 954 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 16:20:54.044083 1380 tablet_service.cc:1467] Processing CreateTablet for tablet 5043f5cac6ac46fd82b828bcbd4a251e (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=8faa0b8064d645dd8ae58a71efec4521]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 16:20:54.044090 1172 tablet_service.cc:1467] Processing CreateTablet for tablet 5043f5cac6ac46fd82b828bcbd4a251e (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=8faa0b8064d645dd8ae58a71efec4521]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 16:20:54.044193 1613 tablet_service.cc:1467] Processing CreateTablet for tablet 5043f5cac6ac46fd82b828bcbd4a251e (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=8faa0b8064d645dd8ae58a71efec4521]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 16:20:54.044263 1172 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5043f5cac6ac46fd82b828bcbd4a251e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:20:54.044272 1380 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5043f5cac6ac46fd82b828bcbd4a251e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:20:54.044334 1613 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5043f5cac6ac46fd82b828bcbd4a251e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:20:54.045557 1762 tablet_bootstrap.cc:492] T 5043f5cac6ac46fd82b828bcbd4a251e P 004c79d477944649ab47d6f916e125fd: Bootstrap starting.
I20260809 16:20:54.045890 1757 tablet_bootstrap.cc:492] T 5043f5cac6ac46fd82b828bcbd4a251e P 9525d3dbf73f46fa93cf72ad9cce21f2: Bootstrap starting.
I20260809 16:20:54.045913 1758 tablet_bootstrap.cc:492] T 5043f5cac6ac46fd82b828bcbd4a251e P fab00ea5697f40b8a24b8c4fdc8084e0: Bootstrap starting.
I20260809 16:20:54.045944 1762 tablet_bootstrap.cc:654] T 5043f5cac6ac46fd82b828bcbd4a251e P 004c79d477944649ab47d6f916e125fd: Neither blocks nor log segments found. Creating new log.
I20260809 16:20:54.046320 1758 tablet_bootstrap.cc:654] T 5043f5cac6ac46fd82b828bcbd4a251e P fab00ea5697f40b8a24b8c4fdc8084e0: Neither blocks nor log segments found. Creating new log.
I20260809 16:20:54.046327 1757 tablet_bootstrap.cc:654] T 5043f5cac6ac46fd82b828bcbd4a251e P 9525d3dbf73f46fa93cf72ad9cce21f2: Neither blocks nor log segments found. Creating new log.
I20260809 16:20:54.046561 1762 tablet_bootstrap.cc:492] T 5043f5cac6ac46fd82b828bcbd4a251e P 004c79d477944649ab47d6f916e125fd: No bootstrap required, opened a new log
I20260809 16:20:54.046609 1762 ts_tablet_manager.cc:1397] T 5043f5cac6ac46fd82b828bcbd4a251e P 004c79d477944649ab47d6f916e125fd: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260809 16:20:54.046731 1762 raft_consensus.cc:348] T 5043f5cac6ac46fd82b828bcbd4a251e P 004c79d477944649ab47d6f916e125fd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fab00ea5697f40b8a24b8c4fdc8084e0" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 33285 } } peers { permanent_uuid: "004c79d477944649ab47d6f916e125fd" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 39867 } } peers { permanent_uuid: "9525d3dbf73f46fa93cf72ad9cce21f2" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 41441 } }
I20260809 16:20:54.046804 1762 raft_consensus.cc:374] T 5043f5cac6ac46fd82b828bcbd4a251e P 004c79d477944649ab47d6f916e125fd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:20:54.046828 1762 raft_consensus.cc:729] T 5043f5cac6ac46fd82b828bcbd4a251e P 004c79d477944649ab47d6f916e125fd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 004c79d477944649ab47d6f916e125fd, State: Initialized, Role: FOLLOWER
I20260809 16:20:54.046886 1762 consensus_queue.cc:260] T 5043f5cac6ac46fd82b828bcbd4a251e P 004c79d477944649ab47d6f916e125fd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fab00ea5697f40b8a24b8c4fdc8084e0" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 33285 } } peers { permanent_uuid: "004c79d477944649ab47d6f916e125fd" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 39867 } } peers { permanent_uuid: "9525d3dbf73f46fa93cf72ad9cce21f2" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 41441 } }
I20260809 16:20:54.046986 1757 tablet_bootstrap.cc:492] T 5043f5cac6ac46fd82b828bcbd4a251e P 9525d3dbf73f46fa93cf72ad9cce21f2: No bootstrap required, opened a new log
I20260809 16:20:54.046990 1762 ts_tablet_manager.cc:1428] T 5043f5cac6ac46fd82b828bcbd4a251e P 004c79d477944649ab47d6f916e125fd: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260809 16:20:54.047039 1757 ts_tablet_manager.cc:1397] T 5043f5cac6ac46fd82b828bcbd4a251e P 9525d3dbf73f46fa93cf72ad9cce21f2: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260809 16:20:54.047037 1794 tablet_replica.cc:439] T 5043f5cac6ac46fd82b828bcbd4a251e P 004c79d477944649ab47d6f916e125fd: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fab00ea5697f40b8a24b8c4fdc8084e0" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 33285 } } peers { permanent_uuid: "004c79d477944649ab47d6f916e125fd" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 39867 } } peers { permanent_uuid: "9525d3dbf73f46fa93cf72ad9cce21f2" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 41441 } } }
I20260809 16:20:54.047116 1794 tablet_replica.cc:442] T 5043f5cac6ac46fd82b828bcbd4a251e P 004c79d477944649ab47d6f916e125fd: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 16:20:54.047197 1757 raft_consensus.cc:348] T 5043f5cac6ac46fd82b828bcbd4a251e P 9525d3dbf73f46fa93cf72ad9cce21f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fab00ea5697f40b8a24b8c4fdc8084e0" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 33285 } } peers { permanent_uuid: "004c79d477944649ab47d6f916e125fd" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 39867 } } peers { permanent_uuid: "9525d3dbf73f46fa93cf72ad9cce21f2" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 41441 } }
I20260809 16:20:54.047283 1757 raft_consensus.cc:374] T 5043f5cac6ac46fd82b828bcbd4a251e P 9525d3dbf73f46fa93cf72ad9cce21f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:20:54.047307 1757 raft_consensus.cc:729] T 5043f5cac6ac46fd82b828bcbd4a251e P 9525d3dbf73f46fa93cf72ad9cce21f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9525d3dbf73f46fa93cf72ad9cce21f2, State: Initialized, Role: FOLLOWER
I20260809 16:20:54.047366 1757 consensus_queue.cc:260] T 5043f5cac6ac46fd82b828bcbd4a251e P 9525d3dbf73f46fa93cf72ad9cce21f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fab00ea5697f40b8a24b8c4fdc8084e0" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 33285 } } peers { permanent_uuid: "004c79d477944649ab47d6f916e125fd" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 39867 } } peers { permanent_uuid: "9525d3dbf73f46fa93cf72ad9cce21f2" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 41441 } }
I20260809 16:20:54.047480 1757 ts_tablet_manager.cc:1428] T 5043f5cac6ac46fd82b828bcbd4a251e P 9525d3dbf73f46fa93cf72ad9cce21f2: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260809 16:20:54.047531 1766 tablet_replica.cc:439] T 5043f5cac6ac46fd82b828bcbd4a251e P 9525d3dbf73f46fa93cf72ad9cce21f2: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fab00ea5697f40b8a24b8c4fdc8084e0" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 33285 } } peers { permanent_uuid: "004c79d477944649ab47d6f916e125fd" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 39867 } } peers { permanent_uuid: "9525d3dbf73f46fa93cf72ad9cce21f2" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 41441 } } }
I20260809 16:20:54.047606 1758 tablet_bootstrap.cc:492] T 5043f5cac6ac46fd82b828bcbd4a251e P fab00ea5697f40b8a24b8c4fdc8084e0: No bootstrap required, opened a new log
I20260809 16:20:54.047616 1766 tablet_replica.cc:442] T 5043f5cac6ac46fd82b828bcbd4a251e P 9525d3dbf73f46fa93cf72ad9cce21f2: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 16:20:54.047648 1758 ts_tablet_manager.cc:1397] T 5043f5cac6ac46fd82b828bcbd4a251e P fab00ea5697f40b8a24b8c4fdc8084e0: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 16:20:54.047799 1758 raft_consensus.cc:348] T 5043f5cac6ac46fd82b828bcbd4a251e P fab00ea5697f40b8a24b8c4fdc8084e0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fab00ea5697f40b8a24b8c4fdc8084e0" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 33285 } } peers { permanent_uuid: "004c79d477944649ab47d6f916e125fd" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 39867 } } peers { permanent_uuid: "9525d3dbf73f46fa93cf72ad9cce21f2" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 41441 } }
I20260809 16:20:54.047870 1758 raft_consensus.cc:374] T 5043f5cac6ac46fd82b828bcbd4a251e P fab00ea5697f40b8a24b8c4fdc8084e0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:20:54.047894 1758 raft_consensus.cc:729] T 5043f5cac6ac46fd82b828bcbd4a251e P fab00ea5697f40b8a24b8c4fdc8084e0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fab00ea5697f40b8a24b8c4fdc8084e0, State: Initialized, Role: FOLLOWER
I20260809 16:20:54.047945 1758 consensus_queue.cc:260] T 5043f5cac6ac46fd82b828bcbd4a251e P fab00ea5697f40b8a24b8c4fdc8084e0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fab00ea5697f40b8a24b8c4fdc8084e0" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 33285 } } peers { permanent_uuid: "004c79d477944649ab47d6f916e125fd" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 39867 } } peers { permanent_uuid: "9525d3dbf73f46fa93cf72ad9cce21f2" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 41441 } }
I20260809 16:20:54.048036 1758 ts_tablet_manager.cc:1428] T 5043f5cac6ac46fd82b828bcbd4a251e P fab00ea5697f40b8a24b8c4fdc8084e0: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260809 16:20:54.048172 1764 tablet_replica.cc:439] T 5043f5cac6ac46fd82b828bcbd4a251e P fab00ea5697f40b8a24b8c4fdc8084e0: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fab00ea5697f40b8a24b8c4fdc8084e0" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 33285 } } peers { permanent_uuid: "004c79d477944649ab47d6f916e125fd" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 39867 } } peers { permanent_uuid: "9525d3dbf73f46fa93cf72ad9cce21f2" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 41441 } } }
I20260809 16:20:54.048259 1764 tablet_replica.cc:442] T 5043f5cac6ac46fd82b828bcbd4a251e P fab00ea5697f40b8a24b8c4fdc8084e0: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 16:20:54.050561 1764 raft_consensus.cc:482] T 5043f5cac6ac46fd82b828bcbd4a251e P fab00ea5697f40b8a24b8c4fdc8084e0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 16:20:54.050624 1764 raft_consensus.cc:504] T 5043f5cac6ac46fd82b828bcbd4a251e P fab00ea5697f40b8a24b8c4fdc8084e0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fab00ea5697f40b8a24b8c4fdc8084e0" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 33285 } } peers { permanent_uuid: "004c79d477944649ab47d6f916e125fd" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 39867 } } peers { permanent_uuid: "9525d3dbf73f46fa93cf72ad9cce21f2" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 41441 } }
I20260809 16:20:54.050755 1764 leader_election.cc:288] T 5043f5cac6ac46fd82b828bcbd4a251e P fab00ea5697f40b8a24b8c4fdc8084e0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 004c79d477944649ab47d6f916e125fd (127.21.51.130:39867), 9525d3dbf73f46fa93cf72ad9cce21f2 (127.21.51.131:41441)
I20260809 16:20:54.050881 1412 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5043f5cac6ac46fd82b828bcbd4a251e" candidate_uuid: "fab00ea5697f40b8a24b8c4fdc8084e0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "004c79d477944649ab47d6f916e125fd" is_pre_election: true
I20260809 16:20:54.050908 1645 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5043f5cac6ac46fd82b828bcbd4a251e" candidate_uuid: "fab00ea5697f40b8a24b8c4fdc8084e0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9525d3dbf73f46fa93cf72ad9cce21f2" is_pre_election: true
I20260809 16:20:54.050963 1412 raft_consensus.cc:2454] T 5043f5cac6ac46fd82b828bcbd4a251e P 004c79d477944649ab47d6f916e125fd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fab00ea5697f40b8a24b8c4fdc8084e0 in term 0.
I20260809 16:20:54.050971 1645 raft_consensus.cc:2454] T 5043f5cac6ac46fd82b828bcbd4a251e P 9525d3dbf73f46fa93cf72ad9cce21f2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fab00ea5697f40b8a24b8c4fdc8084e0 in term 0.
I20260809 16:20:54.051097 1087 leader_election.cc:302] T 5043f5cac6ac46fd82b828bcbd4a251e P fab00ea5697f40b8a24b8c4fdc8084e0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 004c79d477944649ab47d6f916e125fd, fab00ea5697f40b8a24b8c4fdc8084e0; no voters:
I20260809 16:20:54.051200 1764 raft_consensus.cc:2781] T 5043f5cac6ac46fd82b828bcbd4a251e P fab00ea5697f40b8a24b8c4fdc8084e0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 16:20:54.051249 1764 raft_consensus.cc:482] T 5043f5cac6ac46fd82b828bcbd4a251e P fab00ea5697f40b8a24b8c4fdc8084e0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 16:20:54.051273 1764 raft_consensus.cc:3037] T 5043f5cac6ac46fd82b828bcbd4a251e P fab00ea5697f40b8a24b8c4fdc8084e0 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:20:54.051791 1764 raft_consensus.cc:504] T 5043f5cac6ac46fd82b828bcbd4a251e P fab00ea5697f40b8a24b8c4fdc8084e0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fab00ea5697f40b8a24b8c4fdc8084e0" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 33285 } } peers { permanent_uuid: "004c79d477944649ab47d6f916e125fd" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 39867 } } peers { permanent_uuid: "9525d3dbf73f46fa93cf72ad9cce21f2" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 41441 } }
I20260809 16:20:54.051894 1764 leader_election.cc:288] T 5043f5cac6ac46fd82b828bcbd4a251e P fab00ea5697f40b8a24b8c4fdc8084e0 [CANDIDATE]: Term 1 election: Requested vote from peers 004c79d477944649ab47d6f916e125fd (127.21.51.130:39867), 9525d3dbf73f46fa93cf72ad9cce21f2 (127.21.51.131:41441)
I20260809 16:20:54.052037 1412 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5043f5cac6ac46fd82b828bcbd4a251e" candidate_uuid: "fab00ea5697f40b8a24b8c4fdc8084e0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "004c79d477944649ab47d6f916e125fd"
I20260809 16:20:54.052063 1645 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5043f5cac6ac46fd82b828bcbd4a251e" candidate_uuid: "fab00ea5697f40b8a24b8c4fdc8084e0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9525d3dbf73f46fa93cf72ad9cce21f2"
I20260809 16:20:54.052107 1412 raft_consensus.cc:3037] T 5043f5cac6ac46fd82b828bcbd4a251e P 004c79d477944649ab47d6f916e125fd [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:20:54.052130 1645 raft_consensus.cc:3037] T 5043f5cac6ac46fd82b828bcbd4a251e P 9525d3dbf73f46fa93cf72ad9cce21f2 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:20:54.052578 1645 raft_consensus.cc:2454] T 5043f5cac6ac46fd82b828bcbd4a251e P 9525d3dbf73f46fa93cf72ad9cce21f2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fab00ea5697f40b8a24b8c4fdc8084e0 in term 1.
I20260809 16:20:54.052724 1412 raft_consensus.cc:2454] T 5043f5cac6ac46fd82b828bcbd4a251e P 004c79d477944649ab47d6f916e125fd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fab00ea5697f40b8a24b8c4fdc8084e0 in term 1.
I20260809 16:20:54.052788 1089 leader_election.cc:302] T 5043f5cac6ac46fd82b828bcbd4a251e P fab00ea5697f40b8a24b8c4fdc8084e0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9525d3dbf73f46fa93cf72ad9cce21f2, fab00ea5697f40b8a24b8c4fdc8084e0; no voters:
I20260809 16:20:54.052865 1764 raft_consensus.cc:2781] T 5043f5cac6ac46fd82b828bcbd4a251e P fab00ea5697f40b8a24b8c4fdc8084e0 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:20:54.052920 1764 raft_consensus.cc:686] T 5043f5cac6ac46fd82b828bcbd4a251e P fab00ea5697f40b8a24b8c4fdc8084e0 [term 1 LEADER]: Becoming Leader. State: Replica: fab00ea5697f40b8a24b8c4fdc8084e0, State: Running, Role: LEADER
I20260809 16:20:54.052980 1764 consensus_queue.cc:237] T 5043f5cac6ac46fd82b828bcbd4a251e P fab00ea5697f40b8a24b8c4fdc8084e0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fab00ea5697f40b8a24b8c4fdc8084e0" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 33285 } } peers { permanent_uuid: "004c79d477944649ab47d6f916e125fd" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 39867 } } peers { permanent_uuid: "9525d3dbf73f46fa93cf72ad9cce21f2" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 41441 } }
I20260809 16:20:54.053310 1813 tablet_replica.cc:439] T 5043f5cac6ac46fd82b828bcbd4a251e P fab00ea5697f40b8a24b8c4fdc8084e0: TxnStatusTablet state changed. Reason: New leader fab00ea5697f40b8a24b8c4fdc8084e0. Latest consensus state: current_term: 1 leader_uuid: "fab00ea5697f40b8a24b8c4fdc8084e0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fab00ea5697f40b8a24b8c4fdc8084e0" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 33285 } } peers { permanent_uuid: "004c79d477944649ab47d6f916e125fd" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 39867 } } peers { permanent_uuid: "9525d3dbf73f46fa93cf72ad9cce21f2" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 41441 } } }
I20260809 16:20:54.053427 1813 tablet_replica.cc:442] T 5043f5cac6ac46fd82b828bcbd4a251e P fab00ea5697f40b8a24b8c4fdc8084e0: This TxnStatusTablet replica's current role is: LEADER
I20260809 16:20:54.053505 955 catalog_manager.cc:5495] T 5043f5cac6ac46fd82b828bcbd4a251e P fab00ea5697f40b8a24b8c4fdc8084e0 reported cstate change: term changed from 0 to 1, leader changed from <none> to fab00ea5697f40b8a24b8c4fdc8084e0 (127.21.51.129). New cstate: current_term: 1 leader_uuid: "fab00ea5697f40b8a24b8c4fdc8084e0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fab00ea5697f40b8a24b8c4fdc8084e0" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 33285 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "004c79d477944649ab47d6f916e125fd" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 39867 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9525d3dbf73f46fa93cf72ad9cce21f2" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 41441 } health_report { overall_health: UNKNOWN } } }
I20260809 16:20:54.053615 1832 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260809 16:20:54.053723 1832 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260809 16:20:54.061473 1756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:54.058581 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39412 (local address 127.21.51.129:33285)
0809 16:20:54.058627 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0809 16:20:54.058629 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:20:54.058639 (+ 10us) server_negotiation.cc:407] Connection header received
0809 16:20:54.058728 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:20:54.058730 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:20:54.058747 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:20:54.058811 (+ 64us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 16:20:54.058932 (+ 121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:54.059476 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:54.060514 (+ 1038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:54.060814 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:54.060907 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:54.061117 (+ 210us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 16:20:54.061229 (+ 112us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 16:20:54.061316 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0809 16:20:54.061379 (+ 63us) server_negotiation.cc:299] Negotiation successful
0809 16:20:54.061403 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260809 16:20:54.061524 1825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:54.058519 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.129:33285 (local address 127.0.0.1:39412)
0809 16:20:54.058586 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0809 16:20:54.058598 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 16:20:54.058649 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:20:54.058816 (+ 167us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:20:54.058817 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:20:54.058818 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 16:20:54.058874 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:20:54.058876 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:20:54.059558 (+ 682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:20:54.059559 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:20:54.060383 (+ 824us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:20:54.060386 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:20:54.060908 (+ 522us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:20:54.060909 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:20:54.060983 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:54.061005 (+ 22us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 16:20:54.061301 (+ 296us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 16:20:54.061312 (+ 11us) client_negotiation.cc:769] Sending connection context
0809 16:20:54.061373 (+ 61us) client_negotiation.cc:240] Negotiation successful
0809 16:20:54.061426 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31}
I20260809 16:20:54.062640 1645 raft_consensus.cc:1261] T 5043f5cac6ac46fd82b828bcbd4a251e P 9525d3dbf73f46fa93cf72ad9cce21f2 [term 1 FOLLOWER]: Refusing update from remote peer fab00ea5697f40b8a24b8c4fdc8084e0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 16:20:54.062764 1412 raft_consensus.cc:1261] T 5043f5cac6ac46fd82b828bcbd4a251e P 004c79d477944649ab47d6f916e125fd [term 1 FOLLOWER]: Refusing update from remote peer fab00ea5697f40b8a24b8c4fdc8084e0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 16:20:54.063045 1764 consensus_queue.cc:1035] T 5043f5cac6ac46fd82b828bcbd4a251e P fab00ea5697f40b8a24b8c4fdc8084e0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "004c79d477944649ab47d6f916e125fd" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 39867 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 16:20:54.063143 1813 consensus_queue.cc:1035] T 5043f5cac6ac46fd82b828bcbd4a251e P fab00ea5697f40b8a24b8c4fdc8084e0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9525d3dbf73f46fa93cf72ad9cce21f2" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 41441 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 16:20:54.063719 1766 tablet_replica.cc:439] T 5043f5cac6ac46fd82b828bcbd4a251e P 9525d3dbf73f46fa93cf72ad9cce21f2: TxnStatusTablet state changed. Reason: New leader fab00ea5697f40b8a24b8c4fdc8084e0. Latest consensus state: current_term: 1 leader_uuid: "fab00ea5697f40b8a24b8c4fdc8084e0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fab00ea5697f40b8a24b8c4fdc8084e0" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 33285 } } peers { permanent_uuid: "004c79d477944649ab47d6f916e125fd" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 39867 } } peers { permanent_uuid: "9525d3dbf73f46fa93cf72ad9cce21f2" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 41441 } } }
I20260809 16:20:54.063840 1766 tablet_replica.cc:442] T 5043f5cac6ac46fd82b828bcbd4a251e P 9525d3dbf73f46fa93cf72ad9cce21f2: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 16:20:54.063808 1794 tablet_replica.cc:439] T 5043f5cac6ac46fd82b828bcbd4a251e P 004c79d477944649ab47d6f916e125fd: TxnStatusTablet state changed. Reason: New leader fab00ea5697f40b8a24b8c4fdc8084e0. Latest consensus state: current_term: 1 leader_uuid: "fab00ea5697f40b8a24b8c4fdc8084e0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fab00ea5697f40b8a24b8c4fdc8084e0" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 33285 } } peers { permanent_uuid: "004c79d477944649ab47d6f916e125fd" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 39867 } } peers { permanent_uuid: "9525d3dbf73f46fa93cf72ad9cce21f2" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 41441 } } }
I20260809 16:20:54.063894 1794 tablet_replica.cc:442] T 5043f5cac6ac46fd82b828bcbd4a251e P 004c79d477944649ab47d6f916e125fd: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 16:20:54.064653 1764 tablet_replica.cc:439] T 5043f5cac6ac46fd82b828bcbd4a251e P fab00ea5697f40b8a24b8c4fdc8084e0: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "fab00ea5697f40b8a24b8c4fdc8084e0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fab00ea5697f40b8a24b8c4fdc8084e0" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 33285 } } peers { permanent_uuid: "004c79d477944649ab47d6f916e125fd" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 39867 } } peers { permanent_uuid: "9525d3dbf73f46fa93cf72ad9cce21f2" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 41441 } } }
I20260809 16:20:54.064903 1764 tablet_replica.cc:442] T 5043f5cac6ac46fd82b828bcbd4a251e P fab00ea5697f40b8a24b8c4fdc8084e0: This TxnStatusTablet replica's current role is: LEADER
I20260809 16:20:54.064983 1779 mvcc.cc:205] Tried to move back new op lower bound from 7316653891838726144 to 7316653891801276416. Current Snapshot: MvccSnapshot[applied={T|T < 7316653891838726144 or (T in {7316653891838726144})}]
I20260809 16:20:54.065203 1766 tablet_replica.cc:439] T 5043f5cac6ac46fd82b828bcbd4a251e P 9525d3dbf73f46fa93cf72ad9cce21f2: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "fab00ea5697f40b8a24b8c4fdc8084e0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fab00ea5697f40b8a24b8c4fdc8084e0" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 33285 } } peers { permanent_uuid: "004c79d477944649ab47d6f916e125fd" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 39867 } } peers { permanent_uuid: "9525d3dbf73f46fa93cf72ad9cce21f2" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 41441 } } }
I20260809 16:20:54.065297 1766 tablet_replica.cc:442] T 5043f5cac6ac46fd82b828bcbd4a251e P 9525d3dbf73f46fa93cf72ad9cce21f2: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 16:20:54.066133 1813 tablet_replica.cc:439] T 5043f5cac6ac46fd82b828bcbd4a251e P fab00ea5697f40b8a24b8c4fdc8084e0: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "fab00ea5697f40b8a24b8c4fdc8084e0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fab00ea5697f40b8a24b8c4fdc8084e0" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 33285 } } peers { permanent_uuid: "004c79d477944649ab47d6f916e125fd" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 39867 } } peers { permanent_uuid: "9525d3dbf73f46fa93cf72ad9cce21f2" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 41441 } } }
I20260809 16:20:54.066232 1813 tablet_replica.cc:442] T 5043f5cac6ac46fd82b828bcbd4a251e P fab00ea5697f40b8a24b8c4fdc8084e0: This TxnStatusTablet replica's current role is: LEADER
I20260809 16:20:54.066589 1794 tablet_replica.cc:439] T 5043f5cac6ac46fd82b828bcbd4a251e P 004c79d477944649ab47d6f916e125fd: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "fab00ea5697f40b8a24b8c4fdc8084e0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fab00ea5697f40b8a24b8c4fdc8084e0" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 33285 } } peers { permanent_uuid: "004c79d477944649ab47d6f916e125fd" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 39867 } } peers { permanent_uuid: "9525d3dbf73f46fa93cf72ad9cce21f2" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 41441 } } }
I20260809 16:20:54.066668 1794 tablet_replica.cc:442] T 5043f5cac6ac46fd82b828bcbd4a251e P 004c79d477944649ab47d6f916e125fd: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 16:20:54.068584 1756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:54.066289 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39422 (local address 127.21.51.129:33285)
0809 16:20:54.066326 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0809 16:20:54.066328 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:20:54.066335 (+ 7us) server_negotiation.cc:407] Connection header received
0809 16:20:54.066372 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:20:54.066373 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:20:54.066382 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:20:54.066431 (+ 49us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 16:20:54.066547 (+ 116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:54.067239 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:54.067951 (+ 712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:54.068152 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:54.068225 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:54.068362 (+ 137us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 16:20:54.068393 (+ 31us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 16:20:54.068440 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0809 16:20:54.068495 (+ 55us) server_negotiation.cc:299] Negotiation successful
0809 16:20:54.068513 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260809 16:20:54.073704 1756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:54.070991 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.51.129:47143 (local address 127.21.51.129:33285)
0809 16:20:54.071048 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0809 16:20:54.071050 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:20:54.071133 (+ 83us) server_negotiation.cc:407] Connection header received
0809 16:20:54.071213 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:20:54.071214 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:20:54.071221 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:20:54.071259 (+ 38us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 16:20:54.071390 (+ 131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:54.071791 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:54.072935 (+ 1144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:54.073174 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:54.073237 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:54.073466 (+ 229us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 16:20:54.073493 (+ 27us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 16:20:54.073551 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0809 16:20:54.073619 (+ 68us) server_negotiation.cc:299] Negotiation successful
0809 16:20:54.073636 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":41}
I20260809 16:20:54.073794 1840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:54.070910 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.129:33285 (local address 127.21.51.129:47143)
0809 16:20:54.071075 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0809 16:20:54.071087 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 16:20:54.071158 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:20:54.071272 (+ 114us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:20:54.071273 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:20:54.071274 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 16:20:54.071338 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:20:54.071339 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:20:54.071852 (+ 513us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:20:54.071853 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:20:54.072834 (+ 981us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:20:54.072837 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:20:54.073267 (+ 430us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:20:54.073268 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:20:54.073335 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:54.073350 (+ 15us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 16:20:54.073567 (+ 217us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 16:20:54.073569 (+ 2us) client_negotiation.cc:769] Sending connection context
0809 16:20:54.073612 (+ 43us) client_negotiation.cc:240] Negotiation successful
0809 16:20:54.073657 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":90,"threads_started":1}
I20260809 16:20:54.075937 1412 raft_consensus.cc:1261] T 71c481afdbf944d9bca6c12687fd0203 P 004c79d477944649ab47d6f916e125fd [term 1 FOLLOWER]: Refusing update from remote peer fab00ea5697f40b8a24b8c4fdc8084e0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 16:20:54.075980 1645 raft_consensus.cc:1261] T 71c481afdbf944d9bca6c12687fd0203 P 9525d3dbf73f46fa93cf72ad9cce21f2 [term 1 FOLLOWER]: Refusing update from remote peer fab00ea5697f40b8a24b8c4fdc8084e0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 16:20:54.076138 1764 consensus_queue.cc:1035] T 71c481afdbf944d9bca6c12687fd0203 P fab00ea5697f40b8a24b8c4fdc8084e0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "004c79d477944649ab47d6f916e125fd" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 39867 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 16:20:54.076231 1813 consensus_queue.cc:1035] T 71c481afdbf944d9bca6c12687fd0203 P fab00ea5697f40b8a24b8c4fdc8084e0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9525d3dbf73f46fa93cf72ad9cce21f2" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 41441 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 16:20:54.090271 955 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43632:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 16:20:54.090381 955 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43632:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 16:20:54.090986 955 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 40f8427e400f4dab9038e3273f59f1d0: Sending DeleteTablet for 3 replicas of tablet 71c481afdbf944d9bca6c12687fd0203
I20260809 16:20:54.091244 1172 tablet_service.cc:1514] Processing DeleteTablet for tablet 71c481afdbf944d9bca6c12687fd0203 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:20:54 UTC) from {username='oryx', principal='oryx/127.21.51.190@KRBTEST.COM'} at 127.0.0.1:39410
I20260809 16:20:54.091243 1613 tablet_service.cc:1514] Processing DeleteTablet for tablet 71c481afdbf944d9bca6c12687fd0203 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:20:54 UTC) from {username='oryx', principal='oryx/127.21.51.190@KRBTEST.COM'} at 127.0.0.1:37256
I20260809 16:20:54.091337 1801 tablet_replica.cc:330] T 71c481afdbf944d9bca6c12687fd0203 P fab00ea5697f40b8a24b8c4fdc8084e0: stopping tablet replica
I20260809 16:20:54.091338 1380 tablet_service.cc:1514] Processing DeleteTablet for tablet 71c481afdbf944d9bca6c12687fd0203 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:20:54 UTC) from {username='oryx', principal='oryx/127.21.51.190@KRBTEST.COM'} at 127.0.0.1:47044
I20260809 16:20:54.091346 1799 tablet_replica.cc:330] T 71c481afdbf944d9bca6c12687fd0203 P 9525d3dbf73f46fa93cf72ad9cce21f2: stopping tablet replica
I20260809 16:20:54.091391 1801 raft_consensus.cc:2229] T 71c481afdbf944d9bca6c12687fd0203 P fab00ea5697f40b8a24b8c4fdc8084e0 [term 1 LEADER]: Raft consensus shutting down.
I20260809 16:20:54.091408 1799 raft_consensus.cc:2229] T 71c481afdbf944d9bca6c12687fd0203 P 9525d3dbf73f46fa93cf72ad9cce21f2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 16:20:54.091413 1802 tablet_replica.cc:330] T 71c481afdbf944d9bca6c12687fd0203 P 004c79d477944649ab47d6f916e125fd: stopping tablet replica
I20260809 16:20:54.091447 1799 raft_consensus.cc:2258] T 71c481afdbf944d9bca6c12687fd0203 P 9525d3dbf73f46fa93cf72ad9cce21f2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 16:20:54.091456 1802 raft_consensus.cc:2229] T 71c481afdbf944d9bca6c12687fd0203 P 004c79d477944649ab47d6f916e125fd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 16:20:54.091473 1801 raft_consensus.cc:2258] T 71c481afdbf944d9bca6c12687fd0203 P fab00ea5697f40b8a24b8c4fdc8084e0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 16:20:54.091487 1802 raft_consensus.cc:2258] T 71c481afdbf944d9bca6c12687fd0203 P 004c79d477944649ab47d6f916e125fd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 16:20:54.091787 1802 ts_tablet_manager.cc:1905] T 71c481afdbf944d9bca6c12687fd0203 P 004c79d477944649ab47d6f916e125fd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 16:20:54.091862 1799 ts_tablet_manager.cc:1905] T 71c481afdbf944d9bca6c12687fd0203 P 9525d3dbf73f46fa93cf72ad9cce21f2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 16:20:54.091989 1801 ts_tablet_manager.cc:1905] T 71c481afdbf944d9bca6c12687fd0203 P fab00ea5697f40b8a24b8c4fdc8084e0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 16:20:54.093554 1801 ts_tablet_manager.cc:1918] T 71c481afdbf944d9bca6c12687fd0203 P fab00ea5697f40b8a24b8c4fdc8084e0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 16:20:54.093544 1802 ts_tablet_manager.cc:1918] T 71c481afdbf944d9bca6c12687fd0203 P 004c79d477944649ab47d6f916e125fd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 16:20:54.093600 1801 log.cc:1196] T 71c481afdbf944d9bca6c12687fd0203 P fab00ea5697f40b8a24b8c4fdc8084e0: Deleting WAL directory at /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786292417722232-21710-0/minicluster-data/ts-0/wal/wals/71c481afdbf944d9bca6c12687fd0203
I20260809 16:20:54.093601 1802 log.cc:1196] T 71c481afdbf944d9bca6c12687fd0203 P 004c79d477944649ab47d6f916e125fd: Deleting WAL directory at /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786292417722232-21710-0/minicluster-data/ts-1/wal/wals/71c481afdbf944d9bca6c12687fd0203
I20260809 16:20:54.093724 1799 ts_tablet_manager.cc:1918] T 71c481afdbf944d9bca6c12687fd0203 P 9525d3dbf73f46fa93cf72ad9cce21f2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 16:20:54.093783 1799 log.cc:1196] T 71c481afdbf944d9bca6c12687fd0203 P 9525d3dbf73f46fa93cf72ad9cce21f2: Deleting WAL directory at /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786292417722232-21710-0/minicluster-data/ts-2/wal/wals/71c481afdbf944d9bca6c12687fd0203
I20260809 16:20:54.093837 1802 ts_tablet_manager.cc:1939] T 71c481afdbf944d9bca6c12687fd0203 P 004c79d477944649ab47d6f916e125fd: Deleting consensus metadata
I20260809 16:20:54.093842 1801 ts_tablet_manager.cc:1939] T 71c481afdbf944d9bca6c12687fd0203 P fab00ea5697f40b8a24b8c4fdc8084e0: Deleting consensus metadata
I20260809 16:20:54.094020 1799 ts_tablet_manager.cc:1939] T 71c481afdbf944d9bca6c12687fd0203 P 9525d3dbf73f46fa93cf72ad9cce21f2: Deleting consensus metadata
I20260809 16:20:54.094141 934 catalog_manager.cc:4841] TS fab00ea5697f40b8a24b8c4fdc8084e0 (127.21.51.129:33285): tablet 71c481afdbf944d9bca6c12687fd0203 (table test-table [id=aada97400ce84b4e9e5af872bbb0931d]) successfully deleted
I20260809 16:20:54.094161 935 catalog_manager.cc:4841] TS 004c79d477944649ab47d6f916e125fd (127.21.51.130:39867): tablet 71c481afdbf944d9bca6c12687fd0203 (table test-table [id=aada97400ce84b4e9e5af872bbb0931d]) successfully deleted
I20260809 16:20:54.094317 936 catalog_manager.cc:4841] TS 9525d3dbf73f46fa93cf72ad9cce21f2 (127.21.51.131:41441): tablet 71c481afdbf944d9bca6c12687fd0203 (table test-table [id=aada97400ce84b4e9e5af872bbb0931d]) successfully deleted
Aug 09 16:20:54 dist-test-slave-s11t krb5kdc[891](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292454, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 09 16:20:54 dist-test-slave-s11t krb5kdc[891](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.21.51.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 16:20:54 dist-test-slave-s11t krb5kdc[891](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.21.51.190@KRBTEST.COM, Server not found in Kerberos database
W20260809 16:20:54.118014 1034 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:20:54.112239 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43644 (local address 127.21.51.190:37941)
0809 16:20:54.112294 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0809 16:20:54.112296 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:20:54.112463 (+ 167us) server_negotiation.cc:407] Connection header received
0809 16:20:54.112531 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:20:54.112532 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:20:54.112557 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:20:54.112607 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:20:54.113289 (+ 682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:54.113901 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:54.114728 (+ 827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:54.114932 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:54.115004 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:54.117952 (+ 2948us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43644: BlockingRecv error: recv got EOF from 127.0.0.1:43644 (error 108)
Metrics: {"server-negotiator.queue_time_us":29}
Aug 09 16:20:54 dist-test-slave-s11t krb5kdc[891](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292454, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.21.51.190@KRBTEST.COM
I20260809 16:20:54.125949 1034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:54.118772 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43654 (local address 127.21.51.190:37941)
0809 16:20:54.118821 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0809 16:20:54.118823 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:20:54.118985 (+ 162us) server_negotiation.cc:407] Connection header received
0809 16:20:54.119054 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:20:54.119055 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:20:54.119065 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:20:54.119130 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:20:54.119570 (+ 440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:54.120023 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:54.120771 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:54.121073 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:54.121152 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:54.123267 (+ 2115us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:20:54.123269 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:20:54.123269 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:20:54.123276 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:20:54.124890 (+ 1614us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:20:54.125216 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:20:54.125217 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:20:54.125217 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:20:54.125279 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:20:54.125460 (+ 181us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:20:54.125461 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:20:54.125461 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:20:54.125589 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:20:54.125685 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0809 16:20:54.125775 (+ 90us) server_negotiation.cc:299] Negotiation successful
0809 16:20:54.125850 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260809 16:20:54.127545 955 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43654:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 16:20:54.127715 955 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 16:20:54.129153 1172 tablet_service.cc:1467] Processing CreateTablet for tablet 0ac6fc400adb4c6b8fcd494aecbcd0ae (DEFAULT_TABLE table=test-table [id=8994ebd63b0946d1af28560c34e13851]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:20:54.129175 1380 tablet_service.cc:1467] Processing CreateTablet for tablet 0ac6fc400adb4c6b8fcd494aecbcd0ae (DEFAULT_TABLE table=test-table [id=8994ebd63b0946d1af28560c34e13851]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:20:54.129190 1613 tablet_service.cc:1467] Processing CreateTablet for tablet 0ac6fc400adb4c6b8fcd494aecbcd0ae (DEFAULT_TABLE table=test-table [id=8994ebd63b0946d1af28560c34e13851]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:20:54.129290 1172 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0ac6fc400adb4c6b8fcd494aecbcd0ae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:20:54.129297 1380 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0ac6fc400adb4c6b8fcd494aecbcd0ae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:20:54.129315 1613 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0ac6fc400adb4c6b8fcd494aecbcd0ae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:20:54.130586 1758 tablet_bootstrap.cc:492] T 0ac6fc400adb4c6b8fcd494aecbcd0ae P fab00ea5697f40b8a24b8c4fdc8084e0: Bootstrap starting.
I20260809 16:20:54.130602 1762 tablet_bootstrap.cc:492] T 0ac6fc400adb4c6b8fcd494aecbcd0ae P 004c79d477944649ab47d6f916e125fd: Bootstrap starting.
I20260809 16:20:54.130656 1757 tablet_bootstrap.cc:492] T 0ac6fc400adb4c6b8fcd494aecbcd0ae P 9525d3dbf73f46fa93cf72ad9cce21f2: Bootstrap starting.
I20260809 16:20:54.130899 1762 tablet_bootstrap.cc:654] T 0ac6fc400adb4c6b8fcd494aecbcd0ae P 004c79d477944649ab47d6f916e125fd: Neither blocks nor log segments found. Creating new log.
I20260809 16:20:54.130899 1758 tablet_bootstrap.cc:654] T 0ac6fc400adb4c6b8fcd494aecbcd0ae P fab00ea5697f40b8a24b8c4fdc8084e0: Neither blocks nor log segments found. Creating new log.
I20260809 16:20:54.131069 1757 tablet_bootstrap.cc:654] T 0ac6fc400adb4c6b8fcd494aecbcd0ae P 9525d3dbf73f46fa93cf72ad9cce21f2: Neither blocks nor log segments found. Creating new log.
I20260809 16:20:54.131800 1757 tablet_bootstrap.cc:492] T 0ac6fc400adb4c6b8fcd494aecbcd0ae P 9525d3dbf73f46fa93cf72ad9cce21f2: No bootstrap required, opened a new log
I20260809 16:20:54.131834 1762 tablet_bootstrap.cc:492] T 0ac6fc400adb4c6b8fcd494aecbcd0ae P 004c79d477944649ab47d6f916e125fd: No bootstrap required, opened a new log
I20260809 16:20:54.131862 1757 ts_tablet_manager.cc:1397] T 0ac6fc400adb4c6b8fcd494aecbcd0ae P 9525d3dbf73f46fa93cf72ad9cce21f2: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260809 16:20:54.131894 1762 ts_tablet_manager.cc:1397] T 0ac6fc400adb4c6b8fcd494aecbcd0ae P 004c79d477944649ab47d6f916e125fd: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260809 16:20:54.131996 1762 raft_consensus.cc:348] T 0ac6fc400adb4c6b8fcd494aecbcd0ae P 004c79d477944649ab47d6f916e125fd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fab00ea5697f40b8a24b8c4fdc8084e0" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 33285 } } peers { permanent_uuid: "004c79d477944649ab47d6f916e125fd" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 39867 } } peers { permanent_uuid: "9525d3dbf73f46fa93cf72ad9cce21f2" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 41441 } }
I20260809 16:20:54.132071 1762 raft_consensus.cc:374] T 0ac6fc400adb4c6b8fcd494aecbcd0ae P 004c79d477944649ab47d6f916e125fd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:20:54.132030 1757 raft_consensus.cc:348] T 0ac6fc400adb4c6b8fcd494aecbcd0ae P 9525d3dbf73f46fa93cf72ad9cce21f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fab00ea5697f40b8a24b8c4fdc8084e0" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 33285 } } peers { permanent_uuid: "004c79d477944649ab47d6f916e125fd" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 39867 } } peers { permanent_uuid: "9525d3dbf73f46fa93cf72ad9cce21f2" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 41441 } }
I20260809 16:20:54.132099 1762 raft_consensus.cc:729] T 0ac6fc400adb4c6b8fcd494aecbcd0ae P 004c79d477944649ab47d6f916e125fd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 004c79d477944649ab47d6f916e125fd, State: Initialized, Role: FOLLOWER
I20260809 16:20:54.132103 1757 raft_consensus.cc:374] T 0ac6fc400adb4c6b8fcd494aecbcd0ae P 9525d3dbf73f46fa93cf72ad9cce21f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:20:54.132125 1757 raft_consensus.cc:729] T 0ac6fc400adb4c6b8fcd494aecbcd0ae P 9525d3dbf73f46fa93cf72ad9cce21f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9525d3dbf73f46fa93cf72ad9cce21f2, State: Initialized, Role: FOLLOWER
I20260809 16:20:54.132138 1762 consensus_queue.cc:260] T 0ac6fc400adb4c6b8fcd494aecbcd0ae P 004c79d477944649ab47d6f916e125fd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fab00ea5697f40b8a24b8c4fdc8084e0" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 33285 } } peers { permanent_uuid: "004c79d477944649ab47d6f916e125fd" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 39867 } } peers { permanent_uuid: "9525d3dbf73f46fa93cf72ad9cce21f2" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 41441 } }
I20260809 16:20:54.132170 1757 consensus_queue.cc:260] T 0ac6fc400adb4c6b8fcd494aecbcd0ae P 9525d3dbf73f46fa93cf72ad9cce21f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fab00ea5697f40b8a24b8c4fdc8084e0" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 33285 } } peers { permanent_uuid: "004c79d477944649ab47d6f916e125fd" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 39867 } } peers { permanent_uuid: "9525d3dbf73f46fa93cf72ad9cce21f2" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 41441 } }
I20260809 16:20:54.132238 1762 ts_tablet_manager.cc:1428] T 0ac6fc400adb4c6b8fcd494aecbcd0ae P 004c79d477944649ab47d6f916e125fd: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260809 16:20:54.132256 1757 ts_tablet_manager.cc:1428] T 0ac6fc400adb4c6b8fcd494aecbcd0ae P 9525d3dbf73f46fa93cf72ad9cce21f2: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260809 16:20:54.132616 1758 tablet_bootstrap.cc:492] T 0ac6fc400adb4c6b8fcd494aecbcd0ae P fab00ea5697f40b8a24b8c4fdc8084e0: No bootstrap required, opened a new log
I20260809 16:20:54.132668 1758 ts_tablet_manager.cc:1397] T 0ac6fc400adb4c6b8fcd494aecbcd0ae P fab00ea5697f40b8a24b8c4fdc8084e0: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 16:20:54.132829 1758 raft_consensus.cc:348] T 0ac6fc400adb4c6b8fcd494aecbcd0ae P fab00ea5697f40b8a24b8c4fdc8084e0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fab00ea5697f40b8a24b8c4fdc8084e0" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 33285 } } peers { permanent_uuid: "004c79d477944649ab47d6f916e125fd" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 39867 } } peers { permanent_uuid: "9525d3dbf73f46fa93cf72ad9cce21f2" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 41441 } }
I20260809 16:20:54.132901 1758 raft_consensus.cc:374] T 0ac6fc400adb4c6b8fcd494aecbcd0ae P fab00ea5697f40b8a24b8c4fdc8084e0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:20:54.132922 1758 raft_consensus.cc:729] T 0ac6fc400adb4c6b8fcd494aecbcd0ae P fab00ea5697f40b8a24b8c4fdc8084e0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fab00ea5697f40b8a24b8c4fdc8084e0, State: Initialized, Role: FOLLOWER
I20260809 16:20:54.132961 1758 consensus_queue.cc:260] T 0ac6fc400adb4c6b8fcd494aecbcd0ae P fab00ea5697f40b8a24b8c4fdc8084e0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fab00ea5697f40b8a24b8c4fdc8084e0" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 33285 } } peers { permanent_uuid: "004c79d477944649ab47d6f916e125fd" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 39867 } } peers { permanent_uuid: "9525d3dbf73f46fa93cf72ad9cce21f2" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 41441 } }
I20260809 16:20:54.133052 1758 ts_tablet_manager.cc:1428] T 0ac6fc400adb4c6b8fcd494aecbcd0ae P fab00ea5697f40b8a24b8c4fdc8084e0: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 16:20:54.331113 1794 raft_consensus.cc:482] T 0ac6fc400adb4c6b8fcd494aecbcd0ae P 004c79d477944649ab47d6f916e125fd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 16:20:54.331251 1794 raft_consensus.cc:504] T 0ac6fc400adb4c6b8fcd494aecbcd0ae P 004c79d477944649ab47d6f916e125fd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fab00ea5697f40b8a24b8c4fdc8084e0" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 33285 } } peers { permanent_uuid: "004c79d477944649ab47d6f916e125fd" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 39867 } } peers { permanent_uuid: "9525d3dbf73f46fa93cf72ad9cce21f2" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 41441 } }
I20260809 16:20:54.331475 1794 leader_election.cc:288] T 0ac6fc400adb4c6b8fcd494aecbcd0ae P 004c79d477944649ab47d6f916e125fd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fab00ea5697f40b8a24b8c4fdc8084e0 (127.21.51.129:33285), 9525d3dbf73f46fa93cf72ad9cce21f2 (127.21.51.131:41441)
I20260809 16:20:54.331748 1202 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0ac6fc400adb4c6b8fcd494aecbcd0ae" candidate_uuid: "004c79d477944649ab47d6f916e125fd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fab00ea5697f40b8a24b8c4fdc8084e0" is_pre_election: true
I20260809 16:20:54.331872 1202 raft_consensus.cc:2454] T 0ac6fc400adb4c6b8fcd494aecbcd0ae P fab00ea5697f40b8a24b8c4fdc8084e0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 004c79d477944649ab47d6f916e125fd in term 0.
I20260809 16:20:54.331830 1645 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0ac6fc400adb4c6b8fcd494aecbcd0ae" candidate_uuid: "004c79d477944649ab47d6f916e125fd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9525d3dbf73f46fa93cf72ad9cce21f2" is_pre_election: true
I20260809 16:20:54.331956 1645 raft_consensus.cc:2454] T 0ac6fc400adb4c6b8fcd494aecbcd0ae P 9525d3dbf73f46fa93cf72ad9cce21f2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 004c79d477944649ab47d6f916e125fd in term 0.
I20260809 16:20:54.332055 1307 leader_election.cc:302] T 0ac6fc400adb4c6b8fcd494aecbcd0ae P 004c79d477944649ab47d6f916e125fd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 004c79d477944649ab47d6f916e125fd, fab00ea5697f40b8a24b8c4fdc8084e0; no voters:
I20260809 16:20:54.332160 1794 raft_consensus.cc:2781] T 0ac6fc400adb4c6b8fcd494aecbcd0ae P 004c79d477944649ab47d6f916e125fd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 16:20:54.332201 1794 raft_consensus.cc:482] T 0ac6fc400adb4c6b8fcd494aecbcd0ae P 004c79d477944649ab47d6f916e125fd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 16:20:54.332222 1794 raft_consensus.cc:3037] T 0ac6fc400adb4c6b8fcd494aecbcd0ae P 004c79d477944649ab47d6f916e125fd [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:20:54.333096 1794 raft_consensus.cc:504] T 0ac6fc400adb4c6b8fcd494aecbcd0ae P 004c79d477944649ab47d6f916e125fd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fab00ea5697f40b8a24b8c4fdc8084e0" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 33285 } } peers { permanent_uuid: "004c79d477944649ab47d6f916e125fd" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 39867 } } peers { permanent_uuid: "9525d3dbf73f46fa93cf72ad9cce21f2" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 41441 } }
I20260809 16:20:54.333240 1794 leader_election.cc:288] T 0ac6fc400adb4c6b8fcd494aecbcd0ae P 004c79d477944649ab47d6f916e125fd [CANDIDATE]: Term 1 election: Requested vote from peers fab00ea5697f40b8a24b8c4fdc8084e0 (127.21.51.129:33285), 9525d3dbf73f46fa93cf72ad9cce21f2 (127.21.51.131:41441)
I20260809 16:20:54.333431 1202 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0ac6fc400adb4c6b8fcd494aecbcd0ae" candidate_uuid: "004c79d477944649ab47d6f916e125fd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fab00ea5697f40b8a24b8c4fdc8084e0"
I20260809 16:20:54.333431 1645 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0ac6fc400adb4c6b8fcd494aecbcd0ae" candidate_uuid: "004c79d477944649ab47d6f916e125fd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9525d3dbf73f46fa93cf72ad9cce21f2"
I20260809 16:20:54.333508 1202 raft_consensus.cc:3037] T 0ac6fc400adb4c6b8fcd494aecbcd0ae P fab00ea5697f40b8a24b8c4fdc8084e0 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:20:54.333508 1645 raft_consensus.cc:3037] T 0ac6fc400adb4c6b8fcd494aecbcd0ae P 9525d3dbf73f46fa93cf72ad9cce21f2 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:20:54.334308 1645 raft_consensus.cc:2454] T 0ac6fc400adb4c6b8fcd494aecbcd0ae P 9525d3dbf73f46fa93cf72ad9cce21f2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 004c79d477944649ab47d6f916e125fd in term 1.
I20260809 16:20:54.334316 1202 raft_consensus.cc:2454] T 0ac6fc400adb4c6b8fcd494aecbcd0ae P fab00ea5697f40b8a24b8c4fdc8084e0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 004c79d477944649ab47d6f916e125fd in term 1.
I20260809 16:20:54.334519 1310 leader_election.cc:302] T 0ac6fc400adb4c6b8fcd494aecbcd0ae P 004c79d477944649ab47d6f916e125fd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 004c79d477944649ab47d6f916e125fd, 9525d3dbf73f46fa93cf72ad9cce21f2; no voters:
I20260809 16:20:54.334628 1794 raft_consensus.cc:2781] T 0ac6fc400adb4c6b8fcd494aecbcd0ae P 004c79d477944649ab47d6f916e125fd [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:20:54.334717 1794 raft_consensus.cc:686] T 0ac6fc400adb4c6b8fcd494aecbcd0ae P 004c79d477944649ab47d6f916e125fd [term 1 LEADER]: Becoming Leader. State: Replica: 004c79d477944649ab47d6f916e125fd, State: Running, Role: LEADER
I20260809 16:20:54.334774 1794 consensus_queue.cc:237] T 0ac6fc400adb4c6b8fcd494aecbcd0ae P 004c79d477944649ab47d6f916e125fd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fab00ea5697f40b8a24b8c4fdc8084e0" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 33285 } } peers { permanent_uuid: "004c79d477944649ab47d6f916e125fd" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 39867 } } peers { permanent_uuid: "9525d3dbf73f46fa93cf72ad9cce21f2" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 41441 } }
I20260809 16:20:54.335388 955 catalog_manager.cc:5495] T 0ac6fc400adb4c6b8fcd494aecbcd0ae P 004c79d477944649ab47d6f916e125fd reported cstate change: term changed from 0 to 1, leader changed from <none> to 004c79d477944649ab47d6f916e125fd (127.21.51.130). New cstate: current_term: 1 leader_uuid: "004c79d477944649ab47d6f916e125fd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fab00ea5697f40b8a24b8c4fdc8084e0" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 33285 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "004c79d477944649ab47d6f916e125fd" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 39867 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9525d3dbf73f46fa93cf72ad9cce21f2" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 41441 } health_report { overall_health: UNKNOWN } } }
I20260809 16:20:54.359889 1754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:54.357590 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47060 (local address 127.21.51.130:39867)
0809 16:20:54.357640 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0809 16:20:54.357643 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 16:20:54.357652 (+ 9us) server_negotiation.cc:407] Connection header received
0809 16:20:54.357706 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:20:54.357707 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:20:54.357718 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:20:54.357773 (+ 55us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 16:20:54.357940 (+ 167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:54.358427 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:54.359179 (+ 752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:54.359398 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:54.359471 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:54.359634 (+ 163us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 16:20:54.359671 (+ 37us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 16:20:54.359718 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0809 16:20:54.359783 (+ 65us) server_negotiation.cc:299] Negotiation successful
0809 16:20:54.359806 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260809 16:20:54.360774 1202 raft_consensus.cc:1261] T 0ac6fc400adb4c6b8fcd494aecbcd0ae P fab00ea5697f40b8a24b8c4fdc8084e0 [term 1 FOLLOWER]: Refusing update from remote peer 004c79d477944649ab47d6f916e125fd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 16:20:54.360800 1645 raft_consensus.cc:1261] T 0ac6fc400adb4c6b8fcd494aecbcd0ae P 9525d3dbf73f46fa93cf72ad9cce21f2 [term 1 FOLLOWER]: Refusing update from remote peer 004c79d477944649ab47d6f916e125fd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 16:20:54.361047 1771 consensus_queue.cc:1035] T 0ac6fc400adb4c6b8fcd494aecbcd0ae P 004c79d477944649ab47d6f916e125fd [LEADER]: Connected to new peer: Peer: permanent_uuid: "fab00ea5697f40b8a24b8c4fdc8084e0" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 33285 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 16:20:54.361163 1794 consensus_queue.cc:1035] T 0ac6fc400adb4c6b8fcd494aecbcd0ae P 004c79d477944649ab47d6f916e125fd [LEADER]: Connected to new peer: Peer: permanent_uuid: "9525d3dbf73f46fa93cf72ad9cce21f2" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 41441 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 16:20:54.363714 955 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43654:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 16:20:54.363792 955 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43654:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 16:20:54.364352 955 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 40f8427e400f4dab9038e3273f59f1d0: Sending DeleteTablet for 3 replicas of tablet 0ac6fc400adb4c6b8fcd494aecbcd0ae
I20260809 16:20:54.364606 1380 tablet_service.cc:1514] Processing DeleteTablet for tablet 0ac6fc400adb4c6b8fcd494aecbcd0ae with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:20:54 UTC) from {username='oryx', principal='oryx/127.21.51.190@KRBTEST.COM'} at 127.0.0.1:47044
I20260809 16:20:54.364604 1172 tablet_service.cc:1514] Processing DeleteTablet for tablet 0ac6fc400adb4c6b8fcd494aecbcd0ae with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:20:54 UTC) from {username='oryx', principal='oryx/127.21.51.190@KRBTEST.COM'} at 127.0.0.1:39410
I20260809 16:20:54.364646 1613 tablet_service.cc:1514] Processing DeleteTablet for tablet 0ac6fc400adb4c6b8fcd494aecbcd0ae with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:20:54 UTC) from {username='oryx', principal='oryx/127.21.51.190@KRBTEST.COM'} at 127.0.0.1:37256
I20260809 16:20:54.364719 1802 tablet_replica.cc:330] T 0ac6fc400adb4c6b8fcd494aecbcd0ae P 004c79d477944649ab47d6f916e125fd: stopping tablet replica
I20260809 16:20:54.364751 1799 tablet_replica.cc:330] T 0ac6fc400adb4c6b8fcd494aecbcd0ae P 9525d3dbf73f46fa93cf72ad9cce21f2: stopping tablet replica
I20260809 16:20:54.364784 1802 raft_consensus.cc:2229] T 0ac6fc400adb4c6b8fcd494aecbcd0ae P 004c79d477944649ab47d6f916e125fd [term 1 LEADER]: Raft consensus shutting down.
I20260809 16:20:54.364809 1799 raft_consensus.cc:2229] T 0ac6fc400adb4c6b8fcd494aecbcd0ae P 9525d3dbf73f46fa93cf72ad9cce21f2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 16:20:54.364851 1802 raft_consensus.cc:2258] T 0ac6fc400adb4c6b8fcd494aecbcd0ae P 004c79d477944649ab47d6f916e125fd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 16:20:54.364861 1799 raft_consensus.cc:2258] T 0ac6fc400adb4c6b8fcd494aecbcd0ae P 9525d3dbf73f46fa93cf72ad9cce21f2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 16:20:54.364941 1801 tablet_replica.cc:330] T 0ac6fc400adb4c6b8fcd494aecbcd0ae P fab00ea5697f40b8a24b8c4fdc8084e0: stopping tablet replica
I20260809 16:20:54.365002 1801 raft_consensus.cc:2229] T 0ac6fc400adb4c6b8fcd494aecbcd0ae P fab00ea5697f40b8a24b8c4fdc8084e0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 16:20:54.365049 1801 raft_consensus.cc:2258] T 0ac6fc400adb4c6b8fcd494aecbcd0ae P fab00ea5697f40b8a24b8c4fdc8084e0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 16:20:54.364970 955 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43654:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 16:20:54.365286 955 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 16:20:54.365293 1799 ts_tablet_manager.cc:1905] T 0ac6fc400adb4c6b8fcd494aecbcd0ae P 9525d3dbf73f46fa93cf72ad9cce21f2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 16:20:54.365382 1801 ts_tablet_manager.cc:1905] T 0ac6fc400adb4c6b8fcd494aecbcd0ae P fab00ea5697f40b8a24b8c4fdc8084e0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 16:20:54.366335 1380 tablet_service.cc:1467] Processing CreateTablet for tablet 0498fea14ccf4d7d89177fd70a478b37 (DEFAULT_TABLE table=test-table [id=f6ceaeb2bf3c410aad5490a884d0f4fe]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:20:54.366459 1380 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0498fea14ccf4d7d89177fd70a478b37. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:20:54.366612 1613 tablet_service.cc:1467] Processing CreateTablet for tablet 0498fea14ccf4d7d89177fd70a478b37 (DEFAULT_TABLE table=test-table [id=f6ceaeb2bf3c410aad5490a884d0f4fe]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:20:54.366715 1613 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0498fea14ccf4d7d89177fd70a478b37. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:20:54.366981 1799 ts_tablet_manager.cc:1918] T 0ac6fc400adb4c6b8fcd494aecbcd0ae P 9525d3dbf73f46fa93cf72ad9cce21f2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 16:20:54.367025 1799 log.cc:1196] T 0ac6fc400adb4c6b8fcd494aecbcd0ae P 9525d3dbf73f46fa93cf72ad9cce21f2: Deleting WAL directory at /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786292417722232-21710-0/minicluster-data/ts-2/wal/wals/0ac6fc400adb4c6b8fcd494aecbcd0ae
I20260809 16:20:54.367241 1799 ts_tablet_manager.cc:1939] T 0ac6fc400adb4c6b8fcd494aecbcd0ae P 9525d3dbf73f46fa93cf72ad9cce21f2: Deleting consensus metadata
I20260809 16:20:54.367480 1802 ts_tablet_manager.cc:1905] T 0ac6fc400adb4c6b8fcd494aecbcd0ae P 004c79d477944649ab47d6f916e125fd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 16:20:54.367761 1762 tablet_bootstrap.cc:492] T 0498fea14ccf4d7d89177fd70a478b37 P 004c79d477944649ab47d6f916e125fd: Bootstrap starting.
I20260809 16:20:54.367884 1172 tablet_service.cc:1467] Processing CreateTablet for tablet 0498fea14ccf4d7d89177fd70a478b37 (DEFAULT_TABLE table=test-table [id=f6ceaeb2bf3c410aad5490a884d0f4fe]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:20:54.368060 1762 tablet_bootstrap.cc:654] T 0498fea14ccf4d7d89177fd70a478b37 P 004c79d477944649ab47d6f916e125fd: Neither blocks nor log segments found. Creating new log.
I20260809 16:20:54.368070 1757 tablet_bootstrap.cc:492] T 0498fea14ccf4d7d89177fd70a478b37 P 9525d3dbf73f46fa93cf72ad9cce21f2: Bootstrap starting.
I20260809 16:20:54.368229 1801 ts_tablet_manager.cc:1918] T 0ac6fc400adb4c6b8fcd494aecbcd0ae P fab00ea5697f40b8a24b8c4fdc8084e0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 16:20:54.368278 1801 log.cc:1196] T 0ac6fc400adb4c6b8fcd494aecbcd0ae P fab00ea5697f40b8a24b8c4fdc8084e0: Deleting WAL directory at /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786292417722232-21710-0/minicluster-data/ts-0/wal/wals/0ac6fc400adb4c6b8fcd494aecbcd0ae
I20260809 16:20:54.368499 1801 ts_tablet_manager.cc:1939] T 0ac6fc400adb4c6b8fcd494aecbcd0ae P fab00ea5697f40b8a24b8c4fdc8084e0: Deleting consensus metadata
I20260809 16:20:54.368654 1172 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0498fea14ccf4d7d89177fd70a478b37. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:20:54.368705 1762 tablet_bootstrap.cc:492] T 0498fea14ccf4d7d89177fd70a478b37 P 004c79d477944649ab47d6f916e125fd: No bootstrap required, opened a new log
I20260809 16:20:54.368745 1762 ts_tablet_manager.cc:1397] T 0498fea14ccf4d7d89177fd70a478b37 P 004c79d477944649ab47d6f916e125fd: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260809 16:20:54.368816 1757 tablet_bootstrap.cc:654] T 0498fea14ccf4d7d89177fd70a478b37 P 9525d3dbf73f46fa93cf72ad9cce21f2: Neither blocks nor log segments found. Creating new log.
I20260809 16:20:54.368845 1762 raft_consensus.cc:348] T 0498fea14ccf4d7d89177fd70a478b37 P 004c79d477944649ab47d6f916e125fd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fab00ea5697f40b8a24b8c4fdc8084e0" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 33285 } } peers { permanent_uuid: "004c79d477944649ab47d6f916e125fd" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 39867 } } peers { permanent_uuid: "9525d3dbf73f46fa93cf72ad9cce21f2" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 41441 } }
I20260809 16:20:54.368903 1762 raft_consensus.cc:374] T 0498fea14ccf4d7d89177fd70a478b37 P 004c79d477944649ab47d6f916e125fd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:20:54.368921 1762 raft_consensus.cc:729] T 0498fea14ccf4d7d89177fd70a478b37 P 004c79d477944649ab47d6f916e125fd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 004c79d477944649ab47d6f916e125fd, State: Initialized, Role: FOLLOWER
I20260809 16:20:54.368960 1762 consensus_queue.cc:260] T 0498fea14ccf4d7d89177fd70a478b37 P 004c79d477944649ab47d6f916e125fd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fab00ea5697f40b8a24b8c4fdc8084e0" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 33285 } } peers { permanent_uuid: "004c79d477944649ab47d6f916e125fd" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 39867 } } peers { permanent_uuid: "9525d3dbf73f46fa93cf72ad9cce21f2" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 41441 } }
I20260809 16:20:54.369004 1802 ts_tablet_manager.cc:1918] T 0ac6fc400adb4c6b8fcd494aecbcd0ae P 004c79d477944649ab47d6f916e125fd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 16:20:54.369035 1762 ts_tablet_manager.cc:1428] T 0498fea14ccf4d7d89177fd70a478b37 P 004c79d477944649ab47d6f916e125fd: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 16:20:54.369058 1802 log.cc:1196] T 0ac6fc400adb4c6b8fcd494aecbcd0ae P 004c79d477944649ab47d6f916e125fd: Deleting WAL directory at /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786292417722232-21710-0/minicluster-data/ts-1/wal/wals/0ac6fc400adb4c6b8fcd494aecbcd0ae
I20260809 16:20:54.369277 1802 ts_tablet_manager.cc:1939] T 0ac6fc400adb4c6b8fcd494aecbcd0ae P 004c79d477944649ab47d6f916e125fd: Deleting consensus metadata
I20260809 16:20:54.369458 936 catalog_manager.cc:4841] TS 9525d3dbf73f46fa93cf72ad9cce21f2 (127.21.51.131:41441): tablet 0ac6fc400adb4c6b8fcd494aecbcd0ae (table test-table [id=8994ebd63b0946d1af28560c34e13851]) successfully deleted
I20260809 16:20:54.369956 1758 tablet_bootstrap.cc:492] T 0498fea14ccf4d7d89177fd70a478b37 P fab00ea5697f40b8a24b8c4fdc8084e0: Bootstrap starting.
I20260809 16:20:54.369982 1757 tablet_bootstrap.cc:492] T 0498fea14ccf4d7d89177fd70a478b37 P 9525d3dbf73f46fa93cf72ad9cce21f2: No bootstrap required, opened a new log
I20260809 16:20:54.370030 1757 ts_tablet_manager.cc:1397] T 0498fea14ccf4d7d89177fd70a478b37 P 9525d3dbf73f46fa93cf72ad9cce21f2: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260809 16:20:54.370167 935 catalog_manager.cc:4841] TS 004c79d477944649ab47d6f916e125fd (127.21.51.130:39867): tablet 0ac6fc400adb4c6b8fcd494aecbcd0ae (table test-table [id=8994ebd63b0946d1af28560c34e13851]) successfully deleted
I20260809 16:20:54.370178 1757 raft_consensus.cc:348] T 0498fea14ccf4d7d89177fd70a478b37 P 9525d3dbf73f46fa93cf72ad9cce21f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fab00ea5697f40b8a24b8c4fdc8084e0" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 33285 } } peers { permanent_uuid: "004c79d477944649ab47d6f916e125fd" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 39867 } } peers { permanent_uuid: "9525d3dbf73f46fa93cf72ad9cce21f2" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 41441 } }
I20260809 16:20:54.370246 1757 raft_consensus.cc:374] T 0498fea14ccf4d7d89177fd70a478b37 P 9525d3dbf73f46fa93cf72ad9cce21f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:20:54.370276 1757 raft_consensus.cc:729] T 0498fea14ccf4d7d89177fd70a478b37 P 9525d3dbf73f46fa93cf72ad9cce21f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9525d3dbf73f46fa93cf72ad9cce21f2, State: Initialized, Role: FOLLOWER
I20260809 16:20:54.370304 1758 tablet_bootstrap.cc:654] T 0498fea14ccf4d7d89177fd70a478b37 P fab00ea5697f40b8a24b8c4fdc8084e0: Neither blocks nor log segments found. Creating new log.
I20260809 16:20:54.370313 1757 consensus_queue.cc:260] T 0498fea14ccf4d7d89177fd70a478b37 P 9525d3dbf73f46fa93cf72ad9cce21f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fab00ea5697f40b8a24b8c4fdc8084e0" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 33285 } } peers { permanent_uuid: "004c79d477944649ab47d6f916e125fd" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 39867 } } peers { permanent_uuid: "9525d3dbf73f46fa93cf72ad9cce21f2" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 41441 } }
I20260809 16:20:54.370428 1757 ts_tablet_manager.cc:1428] T 0498fea14ccf4d7d89177fd70a478b37 P 9525d3dbf73f46fa93cf72ad9cce21f2: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260809 16:20:54.370810 934 catalog_manager.cc:4841] TS fab00ea5697f40b8a24b8c4fdc8084e0 (127.21.51.129:33285): tablet 0ac6fc400adb4c6b8fcd494aecbcd0ae (table test-table [id=8994ebd63b0946d1af28560c34e13851]) successfully deleted
I20260809 16:20:54.370854 1758 tablet_bootstrap.cc:492] T 0498fea14ccf4d7d89177fd70a478b37 P fab00ea5697f40b8a24b8c4fdc8084e0: No bootstrap required, opened a new log
I20260809 16:20:54.370894 1758 ts_tablet_manager.cc:1397] T 0498fea14ccf4d7d89177fd70a478b37 P fab00ea5697f40b8a24b8c4fdc8084e0: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260809 16:20:54.370997 1758 raft_consensus.cc:348] T 0498fea14ccf4d7d89177fd70a478b37 P fab00ea5697f40b8a24b8c4fdc8084e0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fab00ea5697f40b8a24b8c4fdc8084e0" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 33285 } } peers { permanent_uuid: "004c79d477944649ab47d6f916e125fd" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 39867 } } peers { permanent_uuid: "9525d3dbf73f46fa93cf72ad9cce21f2" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 41441 } }
I20260809 16:20:54.371078 1758 raft_consensus.cc:374] T 0498fea14ccf4d7d89177fd70a478b37 P fab00ea5697f40b8a24b8c4fdc8084e0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:20:54.371114 1758 raft_consensus.cc:729] T 0498fea14ccf4d7d89177fd70a478b37 P fab00ea5697f40b8a24b8c4fdc8084e0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fab00ea5697f40b8a24b8c4fdc8084e0, State: Initialized, Role: FOLLOWER
I20260809 16:20:54.371165 1758 consensus_queue.cc:260] T 0498fea14ccf4d7d89177fd70a478b37 P fab00ea5697f40b8a24b8c4fdc8084e0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fab00ea5697f40b8a24b8c4fdc8084e0" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 33285 } } peers { permanent_uuid: "004c79d477944649ab47d6f916e125fd" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 39867 } } peers { permanent_uuid: "9525d3dbf73f46fa93cf72ad9cce21f2" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 41441 } }
I20260809 16:20:54.371249 1758 ts_tablet_manager.cc:1428] T 0498fea14ccf4d7d89177fd70a478b37 P fab00ea5697f40b8a24b8c4fdc8084e0: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 16:20:54.424435 1813 raft_consensus.cc:482] T 0498fea14ccf4d7d89177fd70a478b37 P fab00ea5697f40b8a24b8c4fdc8084e0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 16:20:54.424525 1813 raft_consensus.cc:504] T 0498fea14ccf4d7d89177fd70a478b37 P fab00ea5697f40b8a24b8c4fdc8084e0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fab00ea5697f40b8a24b8c4fdc8084e0" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 33285 } } peers { permanent_uuid: "004c79d477944649ab47d6f916e125fd" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 39867 } } peers { permanent_uuid: "9525d3dbf73f46fa93cf72ad9cce21f2" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 41441 } }
I20260809 16:20:54.424676 1813 leader_election.cc:288] T 0498fea14ccf4d7d89177fd70a478b37 P fab00ea5697f40b8a24b8c4fdc8084e0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 004c79d477944649ab47d6f916e125fd (127.21.51.130:39867), 9525d3dbf73f46fa93cf72ad9cce21f2 (127.21.51.131:41441)
I20260809 16:20:54.424928 1645 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0498fea14ccf4d7d89177fd70a478b37" candidate_uuid: "fab00ea5697f40b8a24b8c4fdc8084e0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9525d3dbf73f46fa93cf72ad9cce21f2" is_pre_election: true
I20260809 16:20:54.424912 1412 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0498fea14ccf4d7d89177fd70a478b37" candidate_uuid: "fab00ea5697f40b8a24b8c4fdc8084e0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "004c79d477944649ab47d6f916e125fd" is_pre_election: true
I20260809 16:20:54.425027 1645 raft_consensus.cc:2454] T 0498fea14ccf4d7d89177fd70a478b37 P 9525d3dbf73f46fa93cf72ad9cce21f2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fab00ea5697f40b8a24b8c4fdc8084e0 in term 0.
I20260809 16:20:54.425027 1412 raft_consensus.cc:2454] T 0498fea14ccf4d7d89177fd70a478b37 P 004c79d477944649ab47d6f916e125fd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fab00ea5697f40b8a24b8c4fdc8084e0 in term 0.
I20260809 16:20:54.425179 1087 leader_election.cc:302] T 0498fea14ccf4d7d89177fd70a478b37 P fab00ea5697f40b8a24b8c4fdc8084e0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 004c79d477944649ab47d6f916e125fd, fab00ea5697f40b8a24b8c4fdc8084e0; no voters:
I20260809 16:20:54.425316 1813 raft_consensus.cc:2781] T 0498fea14ccf4d7d89177fd70a478b37 P fab00ea5697f40b8a24b8c4fdc8084e0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 16:20:54.425379 1813 raft_consensus.cc:482] T 0498fea14ccf4d7d89177fd70a478b37 P fab00ea5697f40b8a24b8c4fdc8084e0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 16:20:54.425401 1813 raft_consensus.cc:3037] T 0498fea14ccf4d7d89177fd70a478b37 P fab00ea5697f40b8a24b8c4fdc8084e0 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:20:54.426327 1813 raft_consensus.cc:504] T 0498fea14ccf4d7d89177fd70a478b37 P fab00ea5697f40b8a24b8c4fdc8084e0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fab00ea5697f40b8a24b8c4fdc8084e0" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 33285 } } peers { permanent_uuid: "004c79d477944649ab47d6f916e125fd" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 39867 } } peers { permanent_uuid: "9525d3dbf73f46fa93cf72ad9cce21f2" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 41441 } }
I20260809 16:20:54.426499 1813 leader_election.cc:288] T 0498fea14ccf4d7d89177fd70a478b37 P fab00ea5697f40b8a24b8c4fdc8084e0 [CANDIDATE]: Term 1 election: Requested vote from peers 004c79d477944649ab47d6f916e125fd (127.21.51.130:39867), 9525d3dbf73f46fa93cf72ad9cce21f2 (127.21.51.131:41441)
I20260809 16:20:54.426676 1412 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0498fea14ccf4d7d89177fd70a478b37" candidate_uuid: "fab00ea5697f40b8a24b8c4fdc8084e0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "004c79d477944649ab47d6f916e125fd"
I20260809 16:20:54.426662 1645 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0498fea14ccf4d7d89177fd70a478b37" candidate_uuid: "fab00ea5697f40b8a24b8c4fdc8084e0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9525d3dbf73f46fa93cf72ad9cce21f2"
I20260809 16:20:54.426746 1645 raft_consensus.cc:3037] T 0498fea14ccf4d7d89177fd70a478b37 P 9525d3dbf73f46fa93cf72ad9cce21f2 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:20:54.426762 1412 raft_consensus.cc:3037] T 0498fea14ccf4d7d89177fd70a478b37 P 004c79d477944649ab47d6f916e125fd [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:20:54.427392 1412 raft_consensus.cc:2454] T 0498fea14ccf4d7d89177fd70a478b37 P 004c79d477944649ab47d6f916e125fd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fab00ea5697f40b8a24b8c4fdc8084e0 in term 1.
I20260809 16:20:54.427450 1645 raft_consensus.cc:2454] T 0498fea14ccf4d7d89177fd70a478b37 P 9525d3dbf73f46fa93cf72ad9cce21f2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fab00ea5697f40b8a24b8c4fdc8084e0 in term 1.
I20260809 16:20:54.427599 1087 leader_election.cc:302] T 0498fea14ccf4d7d89177fd70a478b37 P fab00ea5697f40b8a24b8c4fdc8084e0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 004c79d477944649ab47d6f916e125fd, fab00ea5697f40b8a24b8c4fdc8084e0; no voters:
I20260809 16:20:54.427716 1813 raft_consensus.cc:2781] T 0498fea14ccf4d7d89177fd70a478b37 P fab00ea5697f40b8a24b8c4fdc8084e0 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:20:54.427798 1813 raft_consensus.cc:686] T 0498fea14ccf4d7d89177fd70a478b37 P fab00ea5697f40b8a24b8c4fdc8084e0 [term 1 LEADER]: Becoming Leader. State: Replica: fab00ea5697f40b8a24b8c4fdc8084e0, State: Running, Role: LEADER
I20260809 16:20:54.427860 1813 consensus_queue.cc:237] T 0498fea14ccf4d7d89177fd70a478b37 P fab00ea5697f40b8a24b8c4fdc8084e0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fab00ea5697f40b8a24b8c4fdc8084e0" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 33285 } } peers { permanent_uuid: "004c79d477944649ab47d6f916e125fd" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 39867 } } peers { permanent_uuid: "9525d3dbf73f46fa93cf72ad9cce21f2" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 41441 } }
I20260809 16:20:54.428431 955 catalog_manager.cc:5495] T 0498fea14ccf4d7d89177fd70a478b37 P fab00ea5697f40b8a24b8c4fdc8084e0 reported cstate change: term changed from 0 to 1, leader changed from <none> to fab00ea5697f40b8a24b8c4fdc8084e0 (127.21.51.129). New cstate: current_term: 1 leader_uuid: "fab00ea5697f40b8a24b8c4fdc8084e0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fab00ea5697f40b8a24b8c4fdc8084e0" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 33285 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "004c79d477944649ab47d6f916e125fd" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 39867 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9525d3dbf73f46fa93cf72ad9cce21f2" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 41441 } health_report { overall_health: UNKNOWN } } }
I20260809 16:20:54.446281 1756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:54.444088 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40012 (local address 127.21.51.129:33285)
0809 16:20:54.444132 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0809 16:20:54.444134 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:20:54.444141 (+ 7us) server_negotiation.cc:407] Connection header received
0809 16:20:54.444218 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:20:54.444219 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:20:54.444229 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:20:54.444278 (+ 49us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 16:20:54.444379 (+ 101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:54.444929 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:54.445632 (+ 703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:54.445825 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:54.445897 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:54.446034 (+ 137us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 16:20:54.446072 (+ 38us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 16:20:54.446129 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0809 16:20:54.446185 (+ 56us) server_negotiation.cc:299] Negotiation successful
0809 16:20:54.446203 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260809 16:20:54.448246 1412 raft_consensus.cc:1261] T 0498fea14ccf4d7d89177fd70a478b37 P 004c79d477944649ab47d6f916e125fd [term 1 FOLLOWER]: Refusing update from remote peer fab00ea5697f40b8a24b8c4fdc8084e0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 16:20:54.448235 1645 raft_consensus.cc:1261] T 0498fea14ccf4d7d89177fd70a478b37 P 9525d3dbf73f46fa93cf72ad9cce21f2 [term 1 FOLLOWER]: Refusing update from remote peer fab00ea5697f40b8a24b8c4fdc8084e0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 16:20:54.448493 1845 consensus_queue.cc:1035] T 0498fea14ccf4d7d89177fd70a478b37 P fab00ea5697f40b8a24b8c4fdc8084e0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9525d3dbf73f46fa93cf72ad9cce21f2" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 41441 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 16:20:54.448601 1813 consensus_queue.cc:1035] T 0498fea14ccf4d7d89177fd70a478b37 P fab00ea5697f40b8a24b8c4fdc8084e0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "004c79d477944649ab47d6f916e125fd" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 39867 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 16:20:54.461519 954 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43654:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 16:20:54.461607 954 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43654:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 16:20:54.462180 954 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 40f8427e400f4dab9038e3273f59f1d0: Sending DeleteTablet for 3 replicas of tablet 0498fea14ccf4d7d89177fd70a478b37
I20260809 16:20:54.462433 1380 tablet_service.cc:1514] Processing DeleteTablet for tablet 0498fea14ccf4d7d89177fd70a478b37 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:20:54 UTC) from {username='oryx', principal='oryx/127.21.51.190@KRBTEST.COM'} at 127.0.0.1:47044
I20260809 16:20:54.462477 1168 tablet_service.cc:1514] Processing DeleteTablet for tablet 0498fea14ccf4d7d89177fd70a478b37 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:20:54 UTC) from {username='oryx', principal='oryx/127.21.51.190@KRBTEST.COM'} at 127.0.0.1:39410
I20260809 16:20:54.462495 1613 tablet_service.cc:1514] Processing DeleteTablet for tablet 0498fea14ccf4d7d89177fd70a478b37 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:20:54 UTC) from {username='oryx', principal='oryx/127.21.51.190@KRBTEST.COM'} at 127.0.0.1:37256
I20260809 16:20:54.462541 1802 tablet_replica.cc:330] T 0498fea14ccf4d7d89177fd70a478b37 P 004c79d477944649ab47d6f916e125fd: stopping tablet replica
I20260809 16:20:54.462584 1799 tablet_replica.cc:330] T 0498fea14ccf4d7d89177fd70a478b37 P 9525d3dbf73f46fa93cf72ad9cce21f2: stopping tablet replica
I20260809 16:20:54.462600 1802 raft_consensus.cc:2229] T 0498fea14ccf4d7d89177fd70a478b37 P 004c79d477944649ab47d6f916e125fd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 16:20:54.462633 1802 raft_consensus.cc:2258] T 0498fea14ccf4d7d89177fd70a478b37 P 004c79d477944649ab47d6f916e125fd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 16:20:54.462646 1799 raft_consensus.cc:2229] T 0498fea14ccf4d7d89177fd70a478b37 P 9525d3dbf73f46fa93cf72ad9cce21f2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 16:20:54.462693 1799 raft_consensus.cc:2258] T 0498fea14ccf4d7d89177fd70a478b37 P 9525d3dbf73f46fa93cf72ad9cce21f2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 16:20:54.462764 1801 tablet_replica.cc:330] T 0498fea14ccf4d7d89177fd70a478b37 P fab00ea5697f40b8a24b8c4fdc8084e0: stopping tablet replica
I20260809 16:20:54.462826 1801 raft_consensus.cc:2229] T 0498fea14ccf4d7d89177fd70a478b37 P fab00ea5697f40b8a24b8c4fdc8084e0 [term 1 LEADER]: Raft consensus shutting down.
I20260809 16:20:54.462888 1801 raft_consensus.cc:2258] T 0498fea14ccf4d7d89177fd70a478b37 P fab00ea5697f40b8a24b8c4fdc8084e0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 16:20:54.463004 1802 ts_tablet_manager.cc:1905] T 0498fea14ccf4d7d89177fd70a478b37 P 004c79d477944649ab47d6f916e125fd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 16:20:54.463060 21710 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrNqUCn/build/release/bin/kudu with pid 1065
I20260809 16:20:54.463186 1801 ts_tablet_manager.cc:1905] T 0498fea14ccf4d7d89177fd70a478b37 P fab00ea5697f40b8a24b8c4fdc8084e0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 16:20:54.464424 1802 ts_tablet_manager.cc:1918] T 0498fea14ccf4d7d89177fd70a478b37 P 004c79d477944649ab47d6f916e125fd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 16:20:54.464469 1802 log.cc:1196] T 0498fea14ccf4d7d89177fd70a478b37 P 004c79d477944649ab47d6f916e125fd: Deleting WAL directory at /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786292417722232-21710-0/minicluster-data/ts-1/wal/wals/0498fea14ccf4d7d89177fd70a478b37
I20260809 16:20:54.464648 1802 ts_tablet_manager.cc:1939] T 0498fea14ccf4d7d89177fd70a478b37 P 004c79d477944649ab47d6f916e125fd: Deleting consensus metadata
I20260809 16:20:54.470340 935 catalog_manager.cc:4841] TS 004c79d477944649ab47d6f916e125fd (127.21.51.130:39867): tablet 0498fea14ccf4d7d89177fd70a478b37 (table test-table [id=f6ceaeb2bf3c410aad5490a884d0f4fe]) successfully deleted
I20260809 16:20:54.475258 1799 ts_tablet_manager.cc:1905] T 0498fea14ccf4d7d89177fd70a478b37 P 9525d3dbf73f46fa93cf72ad9cce21f2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 16:20:54.476867 1799 ts_tablet_manager.cc:1918] T 0498fea14ccf4d7d89177fd70a478b37 P 9525d3dbf73f46fa93cf72ad9cce21f2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 16:20:54.476943 1799 log.cc:1196] T 0498fea14ccf4d7d89177fd70a478b37 P 9525d3dbf73f46fa93cf72ad9cce21f2: Deleting WAL directory at /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786292417722232-21710-0/minicluster-data/ts-2/wal/wals/0498fea14ccf4d7d89177fd70a478b37
I20260809 16:20:54.477140 1799 ts_tablet_manager.cc:1939] T 0498fea14ccf4d7d89177fd70a478b37 P 9525d3dbf73f46fa93cf72ad9cce21f2: Deleting consensus metadata
I20260809 16:20:54.477492 936 catalog_manager.cc:4841] TS 9525d3dbf73f46fa93cf72ad9cce21f2 (127.21.51.131:41441): tablet 0498fea14ccf4d7d89177fd70a478b37 (table test-table [id=f6ceaeb2bf3c410aad5490a884d0f4fe]) successfully deleted
W20260809 16:20:54.482239 934 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.21.51.129:33285 (error 108)
I20260809 16:20:54.482856 1886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:54.482631 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.129:33285 (local address 127.0.0.1:40016)
0809 16:20:54.482739 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0809 16:20:54.482808 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.51.129:33285: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":89,"thread_start_us":42,"threads_started":1}
I20260809 16:20:54.482959 21710 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrNqUCn/build/release/bin/kudu with pid 1283
W20260809 16:20:54.482980 934 catalog_manager.cc:4584] TS fab00ea5697f40b8a24b8c4fdc8084e0 (127.21.51.129:33285): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 0498fea14ccf4d7d89177fd70a478b37: Network error: Client connection negotiation failed: client connection to 127.21.51.129:33285: connect: Connection refused (error 111)
I20260809 16:20:54.502400 21710 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrNqUCn/build/release/bin/kudu with pid 1502
I20260809 16:20:54.512837 21710 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrNqUCn/build/release/bin/kudu with pid 910
2026-08-09T16:20:54Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (2909 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786292417722232-21710-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 16:20:54 dist-test-slave-s11t krb5kdc[1890](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 16:20:54 dist-test-slave-s11t krb5kdc[1890](info): set up 2 sockets
Aug 09 16:20:54 dist-test-slave-s11t krb5kdc[1890](info): commencing operation
krb5kdc: starting...
W20260809 16:20:55.755450 21710 mini_kdc.cc:121] Time spent starting KDC: real 1.214s user 0.005s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786292417722232-21710-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786292417722232-21710-0/krb5kdc/test-user.keytab.
Aug 09 16:20:55 dist-test-slave-s11t krb5kdc[1890](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292455, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T16:20:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T16:20:55Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.51.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.51.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.51.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.190.keytab.
Entry for principal kudu/127.21.51.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.190.keytab.
I20260809 16:20:55.881290 21710 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrNqUCn/build/release/bin/kudu
/tmp/dist-test-taskrNqUCn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786292417722232-21710-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786292417722232-21710-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786292417722232-21710-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786292417722232-21710-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.51.190:41233
--webserver_interface=127.21.51.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.190.keytab
--principal=kudu/127.21.51.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.51.148:33947
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.51.190:41233
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786292417722232-21710-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786292417722232-21710-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786292417722232-21710-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:20:55.962455 1908 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:20:55.962679 1908 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:20:55.962733 1908 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:20:55.963927 1908 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 16:20:55.963984 1908 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:20:55.964007 1908 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:20:55.964028 1908 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 16:20:55.964056 1908 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 16:20:55.965512 1908 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.51.148:33947
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786292417722232-21710-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786292417722232-21710-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.51.190:41233
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.190.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.51.190:41233
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786292417722232-21710-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.51.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1908
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786292417722232-21710-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:20:55.965788 1908 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:20:55.966012 1908 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:20:55.971117 1916 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:20:55.971141 1915 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:20:55.971117 1919 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:20:55.971202 1908 server_base.cc:1034] running on GCE node
I20260809 16:20:55.971617 1908 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:20:55.971961 1908 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:20:55.973125 1908 hybrid_clock.cc:648] HybridClock initialized: now 1786292455973110 us; error 29 us; skew 500 ppm
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-08-09T16:20:55Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1457 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292417722232-21710-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 16:20:56 dist-test-slave-s11t krb5kdc[1930](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 16:20:56 dist-test-slave-s11t krb5kdc[1930](info): set up 2 sockets
Aug 09 16:20:56 dist-test-slave-s11t krb5kdc[1930](info): commencing operation
krb5kdc: starting...
W20260809 16:20:57.262951 21710 mini_kdc.cc:121] Time spent starting KDC: real 1.264s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292417722232-21710-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292417722232-21710-0/krb5kdc/test-user.keytab.
Aug 09 16:20:57 dist-test-slave-s11t krb5kdc[1930](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292457, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T16:20:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T16:20:57Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.51.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.51.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.51.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.190.keytab.
Entry for principal kudu/127.21.51.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.190.keytab.
I20260809 16:20:57.382782 21710 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrNqUCn/build/release/bin/kudu
/tmp/dist-test-taskrNqUCn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292417722232-21710-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292417722232-21710-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292417722232-21710-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292417722232-21710-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.51.190:42871
--webserver_interface=127.21.51.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.190.keytab
--principal=kudu/127.21.51.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.51.148:36277
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.51.190:42871
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292417722232-21710-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292417722232-21710-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292417722232-21710-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:20:57.473764 1950 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:20:57.474018 1950 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:20:57.474059 1950 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:20:57.475314 1950 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 16:20:57.475381 1950 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:20:57.475411 1950 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:20:57.475445 1950 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 16:20:57.475472 1950 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 16:20:57.477164 1950 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.51.148:36277
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292417722232-21710-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292417722232-21710-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.51.190:42871
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.190.keytab
--principal=kudu/127.21.51.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.51.190:42871
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292417722232-21710-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.51.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1950
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292417722232-21710-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:20:57.477468 1950 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:20:57.477710 1950 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:20:57.483039 1958 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:20:57.483160 1950 server_base.cc:1034] running on GCE node
W20260809 16:20:57.483062 1959 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:20:57.483088 1962 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:20:57.483534 1950 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:20:57.483830 1950 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:20:57.484975 1950 hybrid_clock.cc:648] HybridClock initialized: now 1786292457484961 us; error 30 us; skew 500 ppm
Aug 09 16:20:57 dist-test-slave-s11t krb5kdc[1930](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292457, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.51.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:20:57.492676 1950 init.cc:377] Logged in from keytab as kudu/127.21.51.190@KRBTEST.COM (short username kudu)
I20260809 16:20:57.493165 1950 webserver.cc:460] Webserver started at http://127.21.51.190:46603/ using document root <none> and password file <none>
I20260809 16:20:57.493422 1950 fs_manager.cc:359] Metadata directory not provided
I20260809 16:20:57.493489 1950 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:20:57.493635 1950 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:20:57.494726 1950 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292417722232-21710-0/minicluster-data/master-0/data/instance:
uuid: "99a2f6e874df4e72ae8723a75ae18c61"
format_stamp: "Formatted at 2026-08-09 16:20:57 on dist-test-slave-s11t"
server_key: "11c36c14698a4df367d50c3a229bc163"
server_key_iv: "9241ae92f2f44a6fea576be3ae254ec7"
server_key_version: "encryptionkey@0"
I20260809 16:20:57.495105 1950 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292417722232-21710-0/minicluster-data/master-0/wal/instance:
uuid: "99a2f6e874df4e72ae8723a75ae18c61"
format_stamp: "Formatted at 2026-08-09 16:20:57 on dist-test-slave-s11t"
server_key: "11c36c14698a4df367d50c3a229bc163"
server_key_iv: "9241ae92f2f44a6fea576be3ae254ec7"
server_key_version: "encryptionkey@0"
I20260809 16:20:57.496477 1950 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 16:20:57.497334 1979 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:20:57.497532 1950 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260809 16:20:57.497634 1950 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292417722232-21710-0/minicluster-data/master-0/data,/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292417722232-21710-0/minicluster-data/master-0/wal
uuid: "99a2f6e874df4e72ae8723a75ae18c61"
format_stamp: "Formatted at 2026-08-09 16:20:57 on dist-test-slave-s11t"
server_key: "11c36c14698a4df367d50c3a229bc163"
server_key_iv: "9241ae92f2f44a6fea576be3ae254ec7"
server_key_version: "encryptionkey@0"
I20260809 16:20:57.497713 1950 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292417722232-21710-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292417722232-21710-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292417722232-21710-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:20:57.519134 1950 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:20:57.521278 1950 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:20:57.521452 1950 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:20:57.525821 1950 rpc_server.cc:307] RPC server started. Bound to: 127.21.51.190:42871
I20260809 16:20:57.525864 2072 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.51.190:42871 every 8 connection(s)
I20260809 16:20:57.526281 1950 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292417722232-21710-0/minicluster-data/master-0/data/info.pb
I20260809 16:20:57.527159 2073 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:20:57.527948 21710 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrNqUCn/build/release/bin/kudu as pid 1950
I20260809 16:20:57.528044 21710 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292417722232-21710-0/minicluster-data/master-0/wal/instance
I20260809 16:20:57.528265 21710 external_mini_cluster.cc:1428] Setting key 3be9463e43a067d94dff261008b1eb49
I20260809 16:20:57.530409 2073 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 99a2f6e874df4e72ae8723a75ae18c61: Bootstrap starting.
I20260809 16:20:57.531131 2073 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 99a2f6e874df4e72ae8723a75ae18c61: Neither blocks nor log segments found. Creating new log.
I20260809 16:20:57.531457 2073 log.cc:824] T 00000000000000000000000000000000 P 99a2f6e874df4e72ae8723a75ae18c61: Log is configured to *not* fsync() on all Append() calls
I20260809 16:20:57.532645 2073 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 99a2f6e874df4e72ae8723a75ae18c61: No bootstrap required, opened a new log
I20260809 16:20:57.535022 2073 raft_consensus.cc:348] T 00000000000000000000000000000000 P 99a2f6e874df4e72ae8723a75ae18c61 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "99a2f6e874df4e72ae8723a75ae18c61" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 42871 } }
I20260809 16:20:57.535182 2073 raft_consensus.cc:374] T 00000000000000000000000000000000 P 99a2f6e874df4e72ae8723a75ae18c61 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:20:57.535234 2073 raft_consensus.cc:729] T 00000000000000000000000000000000 P 99a2f6e874df4e72ae8723a75ae18c61 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99a2f6e874df4e72ae8723a75ae18c61, State: Initialized, Role: FOLLOWER
I20260809 16:20:57.535367 2073 consensus_queue.cc:260] T 00000000000000000000000000000000 P 99a2f6e874df4e72ae8723a75ae18c61 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "99a2f6e874df4e72ae8723a75ae18c61" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 42871 } }
Aug 09 16:20:57 dist-test-slave-s11t krb5kdc[1930](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292457, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.51.190@KRBTEST.COM
I20260809 16:20:57.535462 2073 raft_consensus.cc:388] T 00000000000000000000000000000000 P 99a2f6e874df4e72ae8723a75ae18c61 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 16:20:57.535509 2073 raft_consensus.cc:482] T 00000000000000000000000000000000 P 99a2f6e874df4e72ae8723a75ae18c61 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 16:20:57.535586 2073 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 99a2f6e874df4e72ae8723a75ae18c61 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:20:57.536326 2073 raft_consensus.cc:504] T 00000000000000000000000000000000 P 99a2f6e874df4e72ae8723a75ae18c61 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "99a2f6e874df4e72ae8723a75ae18c61" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 42871 } }
I20260809 16:20:57.536445 2073 leader_election.cc:302] T 00000000000000000000000000000000 P 99a2f6e874df4e72ae8723a75ae18c61 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 99a2f6e874df4e72ae8723a75ae18c61; no voters:
I20260809 16:20:57.536620 2073 leader_election.cc:288] T 00000000000000000000000000000000 P 99a2f6e874df4e72ae8723a75ae18c61 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 16:20:57.536753 2081 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 99a2f6e874df4e72ae8723a75ae18c61 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:20:57.536953 2081 raft_consensus.cc:686] T 00000000000000000000000000000000 P 99a2f6e874df4e72ae8723a75ae18c61 [term 1 LEADER]: Becoming Leader. State: Replica: 99a2f6e874df4e72ae8723a75ae18c61, State: Running, Role: LEADER
I20260809 16:20:57.536967 2073 sys_catalog.cc:564] T 00000000000000000000000000000000 P 99a2f6e874df4e72ae8723a75ae18c61 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 16:20:57.537177 2081 consensus_queue.cc:237] T 00000000000000000000000000000000 P 99a2f6e874df4e72ae8723a75ae18c61 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "99a2f6e874df4e72ae8723a75ae18c61" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 42871 } }
I20260809 16:20:57.537629 2083 sys_catalog.cc:455] T 00000000000000000000000000000000 P 99a2f6e874df4e72ae8723a75ae18c61 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 99a2f6e874df4e72ae8723a75ae18c61. Latest consensus state: current_term: 1 leader_uuid: "99a2f6e874df4e72ae8723a75ae18c61" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "99a2f6e874df4e72ae8723a75ae18c61" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 42871 } } }
I20260809 16:20:57.537842 2084 sys_catalog.cc:455] T 00000000000000000000000000000000 P 99a2f6e874df4e72ae8723a75ae18c61 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "99a2f6e874df4e72ae8723a75ae18c61" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "99a2f6e874df4e72ae8723a75ae18c61" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 42871 } } }
I20260809 16:20:57.537942 2083 sys_catalog.cc:458] T 00000000000000000000000000000000 P 99a2f6e874df4e72ae8723a75ae18c61 [sys.catalog]: This master's current role is: LEADER
I20260809 16:20:57.538060 2084 sys_catalog.cc:458] T 00000000000000000000000000000000 P 99a2f6e874df4e72ae8723a75ae18c61 [sys.catalog]: This master's current role is: LEADER
I20260809 16:20:57.538254 2099 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 16:20:57.538858 2099 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 16:20:57.540361 2076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:57.528836 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41640 (local address 127.21.51.190:42871)
0809 16:20:57.529037 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0809 16:20:57.529047 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 16:20:57.529081 (+ 34us) server_negotiation.cc:407] Connection header received
0809 16:20:57.529637 (+ 556us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:20:57.529646 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:20:57.529712 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:20:57.529856 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:20:57.530464 (+ 608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:57.531830 (+ 1366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:57.532639 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:57.533162 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:57.533285 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:57.536037 (+ 2752us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:20:57.536041 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:20:57.536041 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:20:57.536052 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:20:57.538403 (+ 2351us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:20:57.538835 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:20:57.538838 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:20:57.538838 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:20:57.538942 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:20:57.539257 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:20:57.539257 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:20:57.539258 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:20:57.539588 (+ 330us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:20:57.539750 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0809 16:20:57.539924 (+ 174us) server_negotiation.cc:299] Negotiation successful
0809 16:20:57.540190 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":149,"thread_start_us":85,"threads_started":1}
I20260809 16:20:57.540980 2099 catalog_manager.cc:1347] Generated new cluster ID: 6575424e3c774195aa86544c5545bfa4
I20260809 16:20:57.541035 2099 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 16:20:57.552518 2099 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 16:20:57.553354 2099 catalog_manager.cc:1504] Loading token signing keys...
I20260809 16:20:57.556977 2099 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 99a2f6e874df4e72ae8723a75ae18c61: Generated new TSK 0
I20260809 16:20:57.557173 2099 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.21.51.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.51.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.51.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.129.keytab.
Entry for principal kudu/127.21.51.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.129.keytab.
I20260809 16:20:57.591416 21710 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrNqUCn/build/release/bin/kudu
/tmp/dist-test-taskrNqUCn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292417722232-21710-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292417722232-21710-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292417722232-21710-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292417722232-21710-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.51.129:0
--local_ip_for_outbound_sockets=127.21.51.129
--webserver_interface=127.21.51.129
--webserver_port=0
--tserver_master_addrs=127.21.51.190:42871
--keytab_file=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.129.keytab
--principal=kudu/127.21.51.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.51.148:36277
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292417722232-21710-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292417722232-21710-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292417722232-21710-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:20:57.698073 2114 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:20:57.698267 2114 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:20:57.698289 2114 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:20:57.699419 2114 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:20:57.699461 2114 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:20:57.699486 2114 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.51.129
I20260809 16:20:57.700999 2114 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.51.148:36277
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292417722232-21710-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292417722232-21710-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.129.keytab
--principal=kudu/127.21.51.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.51.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292417722232-21710-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.51.129
--webserver_port=0
--tserver_master_addrs=127.21.51.190:42871
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2114
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.51.129
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292417722232-21710-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:20:57.701244 2114 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:20:57.701524 2114 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:20:57.706653 2130 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:20:57.706655 2122 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:20:57.706833 2114 server_base.cc:1034] running on GCE node
W20260809 16:20:57.706701 2127 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:20:57.707144 2114 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:20:57.707437 2114 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:20:57.708580 2114 hybrid_clock.cc:648] HybridClock initialized: now 1786292457708563 us; error 42 us; skew 500 ppm
Aug 09 16:20:57 dist-test-slave-s11t krb5kdc[1930](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292457, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.51.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:20:57.711162 2114 init.cc:377] Logged in from keytab as kudu/127.21.51.129@KRBTEST.COM (short username kudu)
I20260809 16:20:57.711599 2114 webserver.cc:460] Webserver started at http://127.21.51.129:44661/ using document root <none> and password file <none>
I20260809 16:20:57.711831 2114 fs_manager.cc:359] Metadata directory not provided
I20260809 16:20:57.711896 2114 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:20:57.712047 2114 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:20:57.713157 2114 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292417722232-21710-0/minicluster-data/ts-0/data/instance:
uuid: "af2a3c461ca64de3bdd776680c9cb4ae"
format_stamp: "Formatted at 2026-08-09 16:20:57 on dist-test-slave-s11t"
server_key: "755b809679d54bbdb6078ea3896cc904"
server_key_iv: "805ad5099c30ab74d63639bdfdeded39"
server_key_version: "encryptionkey@0"
I20260809 16:20:57.713517 2114 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292417722232-21710-0/minicluster-data/ts-0/wal/instance:
uuid: "af2a3c461ca64de3bdd776680c9cb4ae"
format_stamp: "Formatted at 2026-08-09 16:20:57 on dist-test-slave-s11t"
server_key: "755b809679d54bbdb6078ea3896cc904"
server_key_iv: "805ad5099c30ab74d63639bdfdeded39"
server_key_version: "encryptionkey@0"
I20260809 16:20:57.714855 2114 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 16:20:57.715559 2140 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:20:57.715728 2114 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 16:20:57.715830 2114 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292417722232-21710-0/minicluster-data/ts-0/data,/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292417722232-21710-0/minicluster-data/ts-0/wal
uuid: "af2a3c461ca64de3bdd776680c9cb4ae"
format_stamp: "Formatted at 2026-08-09 16:20:57 on dist-test-slave-s11t"
server_key: "755b809679d54bbdb6078ea3896cc904"
server_key_iv: "805ad5099c30ab74d63639bdfdeded39"
server_key_version: "encryptionkey@0"
I20260809 16:20:57.715903 2114 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292417722232-21710-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292417722232-21710-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292417722232-21710-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:20:57.728876 2114 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:20:57.731118 2114 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:20:57.731281 2114 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:20:57.731499 2114 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:20:57.731806 2114 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:20:57.731864 2114 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:20:57.731897 2114 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:20:57.731930 2114 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:20:57.737692 2114 rpc_server.cc:307] RPC server started. Bound to: 127.21.51.129:32823
I20260809 16:20:57.737756 2343 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.51.129:32823 every 8 connection(s)
I20260809 16:20:57.738066 2114 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292417722232-21710-0/minicluster-data/ts-0/data/info.pb
Aug 09 16:20:57 dist-test-slave-s11t krb5kdc[1930](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292457, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.51.129@KRBTEST.COM for kudu/127.21.51.190@KRBTEST.COM
I20260809 16:20:57.746891 2076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:57.738804 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.51.129:38497 (local address 127.21.51.190:42871)
0809 16:20:57.738872 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0809 16:20:57.738875 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 16:20:57.739102 (+ 227us) server_negotiation.cc:407] Connection header received
0809 16:20:57.739574 (+ 472us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:20:57.739575 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:20:57.739585 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:20:57.739633 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:20:57.740498 (+ 865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:57.741076 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:57.742024 (+ 948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:57.742217 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:57.742266 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:57.744416 (+ 2150us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:20:57.744417 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:20:57.744417 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:20:57.744424 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:20:57.745684 (+ 1260us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:20:57.746075 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:20:57.746077 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:20:57.746077 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:20:57.746139 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:20:57.746355 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:20:57.746355 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:20:57.746356 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:20:57.746466 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:20:57.746520 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0809 16:20:57.746741 (+ 221us) server_negotiation.cc:299] Negotiation successful
0809 16:20:57.746826 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260809 16:20:57.747037 2347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:57.738806 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:42871 (local address 127.21.51.129:38497)
0809 16:20:57.738989 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0809 16:20:57.739024 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0809 16:20:57.739494 (+ 470us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:20:57.739691 (+ 197us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:20:57.739695 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:20:57.740235 (+ 540us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:20:57.740427 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:20:57.740431 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:20:57.741152 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:20:57.741153 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:20:57.741953 (+ 800us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:20:57.741955 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:20:57.742285 (+ 330us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:20:57.742286 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:20:57.742403 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:57.742911 (+ 508us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:20:57.742926 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:20:57.744323 (+ 1397us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:20:57.745756 (+ 1433us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:20:57.745762 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:20:57.745763 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:20:57.745984 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:20:57.746205 (+ 221us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:20:57.746206 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:20:57.746206 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:20:57.746299 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:20:57.746519 (+ 220us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:20:57.746523 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:20:57.746630 (+ 107us) client_negotiation.cc:769] Sending connection context
0809 16:20:57.746710 (+ 80us) client_negotiation.cc:240] Negotiation successful
0809 16:20:57.746831 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":104,"threads_started":1}
I20260809 16:20:57.747607 2345 heartbeater.cc:347] Connected to a master server at 127.21.51.190:42871
I20260809 16:20:57.747697 2345 heartbeater.cc:464] Registering TS with master...
I20260809 16:20:57.747903 2345 heartbeater.cc:511] Master 127.21.51.190:42871 requested a full tablet report, sending...
I20260809 16:20:57.748248 21710 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrNqUCn/build/release/bin/kudu as pid 2114
I20260809 16:20:57.748371 21710 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292417722232-21710-0/minicluster-data/ts-0/wal/instance
I20260809 16:20:57.748566 21710 external_mini_cluster.cc:1428] Setting key 5f71aabc53ff61979c2da489a346e32e
I20260809 16:20:57.748582 2005 ts_manager.cc:194] Registered new tserver with Master: af2a3c461ca64de3bdd776680c9cb4ae (127.21.51.129:32823)
I20260809 16:20:57.749598 2005 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.51.129@KRBTEST.COM'} at 127.21.51.129:38497
WARNING: no policy specified for kudu/127.21.51.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.51.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.51.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.130.keytab.
Entry for principal kudu/127.21.51.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.130.keytab.
I20260809 16:20:57.773597 21710 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrNqUCn/build/release/bin/kudu
/tmp/dist-test-taskrNqUCn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292417722232-21710-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292417722232-21710-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292417722232-21710-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292417722232-21710-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.51.130:0
--local_ip_for_outbound_sockets=127.21.51.130
--webserver_interface=127.21.51.130
--webserver_port=0
--tserver_master_addrs=127.21.51.190:42871
--keytab_file=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.130.keytab
--principal=kudu/127.21.51.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.51.148:36277
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292417722232-21710-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292417722232-21710-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292417722232-21710-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:20:57.855407 2352 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:20:57.855602 2352 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:20:57.855624 2352 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:20:57.856741 2352 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:20:57.856778 2352 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:20:57.856804 2352 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.51.130
I20260809 16:20:57.858157 2352 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.51.148:36277
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292417722232-21710-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292417722232-21710-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.130.keytab
--principal=kudu/127.21.51.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.51.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292417722232-21710-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.51.130
--webserver_port=0
--tserver_master_addrs=127.21.51.190:42871
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2352
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.51.130
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292417722232-21710-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:20:57.858343 2352 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:20:57.858520 2352 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:20:57.863549 2366 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:20:57.863566 2352 server_base.cc:1034] running on GCE node
W20260809 16:20:57.863597 2361 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:20:57.863550 2363 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:20:57.864032 2352 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:20:57.864281 2352 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:20:57.865479 2352 hybrid_clock.cc:648] HybridClock initialized: now 1786292457865461 us; error 30 us; skew 500 ppm
Aug 09 16:20:57 dist-test-slave-s11t krb5kdc[1930](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292457, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.51.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:20:57.867973 2352 init.cc:377] Logged in from keytab as kudu/127.21.51.130@KRBTEST.COM (short username kudu)
I20260809 16:20:57.868479 2352 webserver.cc:460] Webserver started at http://127.21.51.130:39677/ using document root <none> and password file <none>
I20260809 16:20:57.868769 2352 fs_manager.cc:359] Metadata directory not provided
I20260809 16:20:57.868837 2352 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:20:57.868974 2352 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:20:57.869977 2352 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292417722232-21710-0/minicluster-data/ts-1/data/instance:
uuid: "30adb6222bbd46ea8a9114064e90739b"
format_stamp: "Formatted at 2026-08-09 16:20:57 on dist-test-slave-s11t"
server_key: "6dc493fcd2b35a6f86d0563818bf5e49"
server_key_iv: "57fe9078a2c8e3a31940e35e5ba65f78"
server_key_version: "encryptionkey@0"
I20260809 16:20:57.870343 2352 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292417722232-21710-0/minicluster-data/ts-1/wal/instance:
uuid: "30adb6222bbd46ea8a9114064e90739b"
format_stamp: "Formatted at 2026-08-09 16:20:57 on dist-test-slave-s11t"
server_key: "6dc493fcd2b35a6f86d0563818bf5e49"
server_key_iv: "57fe9078a2c8e3a31940e35e5ba65f78"
server_key_version: "encryptionkey@0"
I20260809 16:20:57.871654 2352 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 16:20:57.872406 2375 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:20:57.872567 2352 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 16:20:57.872664 2352 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292417722232-21710-0/minicluster-data/ts-1/data,/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292417722232-21710-0/minicluster-data/ts-1/wal
uuid: "30adb6222bbd46ea8a9114064e90739b"
format_stamp: "Formatted at 2026-08-09 16:20:57 on dist-test-slave-s11t"
server_key: "6dc493fcd2b35a6f86d0563818bf5e49"
server_key_iv: "57fe9078a2c8e3a31940e35e5ba65f78"
server_key_version: "encryptionkey@0"
I20260809 16:20:57.872797 2352 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292417722232-21710-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292417722232-21710-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292417722232-21710-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:20:57.889024 2352 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:20:57.890769 2352 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:20:57.890923 2352 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:20:57.891132 2352 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:20:57.891431 2352 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:20:57.891485 2352 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:20:57.891527 2352 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:20:57.891552 2352 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:20:57.897536 2352 rpc_server.cc:307] RPC server started. Bound to: 127.21.51.130:46527
I20260809 16:20:57.897580 2565 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.51.130:46527 every 8 connection(s)
I20260809 16:20:57.897886 2352 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292417722232-21710-0/minicluster-data/ts-1/data/info.pb
I20260809 16:20:57.898404 21710 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrNqUCn/build/release/bin/kudu as pid 2352
I20260809 16:20:57.898512 21710 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292417722232-21710-0/minicluster-data/ts-1/wal/instance
I20260809 16:20:57.898768 21710 external_mini_cluster.cc:1428] Setting key 47eeb9d6f8997045acfa7c1232957463
Aug 09 16:20:57 dist-test-slave-s11t krb5kdc[1930](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292457, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.51.130@KRBTEST.COM for kudu/127.21.51.190@KRBTEST.COM
I20260809 16:20:57.908656 2076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:57.898589 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.51.130:56193 (local address 127.21.51.190:42871)
0809 16:20:57.898678 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0809 16:20:57.898681 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 16:20:57.898910 (+ 229us) server_negotiation.cc:407] Connection header received
0809 16:20:57.899719 (+ 809us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:20:57.899720 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:20:57.899733 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:20:57.899779 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:20:57.901083 (+ 1304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:57.901619 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:57.902621 (+ 1002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:57.902954 (+ 333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:57.903027 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:57.905503 (+ 2476us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:20:57.905504 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:20:57.905505 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:20:57.905515 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:20:57.907178 (+ 1663us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:20:57.907585 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:20:57.907586 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:20:57.907587 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:20:57.907657 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:20:57.907908 (+ 251us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:20:57.907909 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:20:57.907909 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:20:57.908058 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:20:57.908142 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0809 16:20:57.908451 (+ 309us) server_negotiation.cc:299] Negotiation successful
0809 16:20:57.908549 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260809 16:20:57.908679 2570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:57.898558 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:42871 (local address 127.21.51.130:56193)
0809 16:20:57.898762 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0809 16:20:57.898810 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0809 16:20:57.899591 (+ 781us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:20:57.899853 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:20:57.899858 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:20:57.900728 (+ 870us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:20:57.900995 (+ 267us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:20:57.901002 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:20:57.901715 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:20:57.901716 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:20:57.902527 (+ 811us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:20:57.902530 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:20:57.903052 (+ 522us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:20:57.903053 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:20:57.903198 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:57.903706 (+ 508us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:20:57.903716 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:20:57.905404 (+ 1688us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:20:57.907268 (+ 1864us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:20:57.907274 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:20:57.907276 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:20:57.907505 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:20:57.907733 (+ 228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:20:57.907734 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:20:57.907734 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:20:57.907832 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:20:57.908149 (+ 317us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:20:57.908155 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:20:57.908303 (+ 148us) client_negotiation.cc:769] Sending connection context
0809 16:20:57.908400 (+ 97us) client_negotiation.cc:240] Negotiation successful
0809 16:20:57.908522 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":106,"threads_started":1}
WARNING: no policy specified for kudu/127.21.51.131@KRBTEST.COM; defaulting to no policy
I20260809 16:20:57.909449 2567 heartbeater.cc:347] Connected to a master server at 127.21.51.190:42871
I20260809 16:20:57.909565 2567 heartbeater.cc:464] Registering TS with master...
I20260809 16:20:57.909770 2567 heartbeater.cc:511] Master 127.21.51.190:42871 requested a full tablet report, sending...
I20260809 16:20:57.910235 2005 ts_manager.cc:194] Registered new tserver with Master: 30adb6222bbd46ea8a9114064e90739b (127.21.51.130:46527)
I20260809 16:20:57.910586 2005 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.51.130@KRBTEST.COM'} at 127.21.51.130:56193
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.51.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.51.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.131.keytab.
Entry for principal kudu/127.21.51.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.131.keytab.
I20260809 16:20:57.926967 21710 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrNqUCn/build/release/bin/kudu
/tmp/dist-test-taskrNqUCn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292417722232-21710-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292417722232-21710-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292417722232-21710-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292417722232-21710-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.51.131:0
--local_ip_for_outbound_sockets=127.21.51.131
--webserver_interface=127.21.51.131
--webserver_port=0
--tserver_master_addrs=127.21.51.190:42871
--keytab_file=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.131.keytab
--principal=kudu/127.21.51.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.51.148:36277
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292417722232-21710-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292417722232-21710-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292417722232-21710-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:20:58.009912 2574 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:20:58.010121 2574 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:20:58.010144 2574 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:20:58.011220 2574 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:20:58.011255 2574 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:20:58.011281 2574 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.51.131
I20260809 16:20:58.012769 2574 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.51.148:36277
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292417722232-21710-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292417722232-21710-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.131.keytab
--principal=kudu/127.21.51.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.51.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292417722232-21710-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.51.131
--webserver_port=0
--tserver_master_addrs=127.21.51.190:42871
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2574
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.51.131
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292417722232-21710-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:20:58.013032 2574 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:20:58.013234 2574 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:20:58.018118 2579 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:20:58.018138 2580 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:20:58.018124 2583 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:20:58.018455 2574 server_base.cc:1034] running on GCE node
I20260809 16:20:58.018636 2574 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:20:58.018920 2574 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:20:58.020080 2574 hybrid_clock.cc:648] HybridClock initialized: now 1786292458020074 us; error 36 us; skew 500 ppm
Aug 09 16:20:58 dist-test-slave-s11t krb5kdc[1930](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292458, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.51.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:20:58.022722 2574 init.cc:377] Logged in from keytab as kudu/127.21.51.131@KRBTEST.COM (short username kudu)
I20260809 16:20:58.023187 2574 webserver.cc:460] Webserver started at http://127.21.51.131:46711/ using document root <none> and password file <none>
I20260809 16:20:58.023370 2574 fs_manager.cc:359] Metadata directory not provided
I20260809 16:20:58.023412 2574 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:20:58.023499 2574 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:20:58.024473 2574 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292417722232-21710-0/minicluster-data/ts-2/data/instance:
uuid: "88326b268b8e485ab4d425ae71ebfc2d"
format_stamp: "Formatted at 2026-08-09 16:20:58 on dist-test-slave-s11t"
server_key: "5c0561e84b6f92beb5e7db9ee107b109"
server_key_iv: "1ca708d63c0980734722361b9f51a026"
server_key_version: "encryptionkey@0"
I20260809 16:20:58.024922 2574 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292417722232-21710-0/minicluster-data/ts-2/wal/instance:
uuid: "88326b268b8e485ab4d425ae71ebfc2d"
format_stamp: "Formatted at 2026-08-09 16:20:58 on dist-test-slave-s11t"
server_key: "5c0561e84b6f92beb5e7db9ee107b109"
server_key_iv: "1ca708d63c0980734722361b9f51a026"
server_key_version: "encryptionkey@0"
I20260809 16:20:58.026191 2574 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:20:58.026881 2591 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:20:58.027063 2574 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 16:20:58.027169 2574 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292417722232-21710-0/minicluster-data/ts-2/data,/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292417722232-21710-0/minicluster-data/ts-2/wal
uuid: "88326b268b8e485ab4d425ae71ebfc2d"
format_stamp: "Formatted at 2026-08-09 16:20:58 on dist-test-slave-s11t"
server_key: "5c0561e84b6f92beb5e7db9ee107b109"
server_key_iv: "1ca708d63c0980734722361b9f51a026"
server_key_version: "encryptionkey@0"
I20260809 16:20:58.027240 2574 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292417722232-21710-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292417722232-21710-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292417722232-21710-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:20:58.059737 2574 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:20:58.061854 2574 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:20:58.062021 2574 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:20:58.062247 2574 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:20:58.062597 2574 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:20:58.062651 2574 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:20:58.062705 2574 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:20:58.062748 2574 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:20:58.068531 2574 rpc_server.cc:307] RPC server started. Bound to: 127.21.51.131:36787
I20260809 16:20:58.068557 2777 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.51.131:36787 every 8 connection(s)
I20260809 16:20:58.068979 2574 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292417722232-21710-0/minicluster-data/ts-2/data/info.pb
I20260809 16:20:58.072525 21710 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrNqUCn/build/release/bin/kudu as pid 2574
I20260809 16:20:58.072628 21710 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292417722232-21710-0/minicluster-data/ts-2/wal/instance
I20260809 16:20:58.072860 21710 external_mini_cluster.cc:1428] Setting key 762f4bc26145b8949fcdf1b4cb2d9b23
Aug 09 16:20:58 dist-test-slave-s11t krb5kdc[1930](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292458, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.51.131@KRBTEST.COM for kudu/127.21.51.190@KRBTEST.COM
I20260809 16:20:58.078512 2076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:58.069719 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.51.131:45495 (local address 127.21.51.190:42871)
0809 16:20:58.069796 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0809 16:20:58.069799 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 16:20:58.069983 (+ 184us) server_negotiation.cc:407] Connection header received
0809 16:20:58.070467 (+ 484us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:20:58.070468 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:20:58.070478 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:20:58.070520 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:20:58.071402 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:58.071906 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:58.073368 (+ 1462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:58.073670 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:58.073724 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:58.075951 (+ 2227us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:20:58.075952 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:20:58.075953 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:20:58.075959 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:20:58.077366 (+ 1407us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:20:58.077749 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:20:58.077750 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:20:58.077750 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:20:58.077804 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:20:58.078012 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:20:58.078012 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:20:58.078012 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:20:58.078111 (+ 99us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:20:58.078162 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0809 16:20:58.078389 (+ 227us) server_negotiation.cc:299] Negotiation successful
0809 16:20:58.078449 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260809 16:20:58.078675 2785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:58.069682 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:42871 (local address 127.21.51.131:45495)
0809 16:20:58.069867 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0809 16:20:58.069903 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0809 16:20:58.070372 (+ 469us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:20:58.070571 (+ 199us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:20:58.070574 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:20:58.071130 (+ 556us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:20:58.071323 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:20:58.071327 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:20:58.071990 (+ 663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:20:58.071991 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:20:58.073290 (+ 1299us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:20:58.073292 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:20:58.073741 (+ 449us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:20:58.073743 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:20:58.073897 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:58.074422 (+ 525us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:20:58.074436 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:20:58.075854 (+ 1418us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:20:58.077443 (+ 1589us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:20:58.077449 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:20:58.077451 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:20:58.077682 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:20:58.077855 (+ 173us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:20:58.077856 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:20:58.077856 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:20:58.077959 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:20:58.078165 (+ 206us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:20:58.078169 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:20:58.078282 (+ 113us) client_negotiation.cc:769] Sending connection context
0809 16:20:58.078364 (+ 82us) client_negotiation.cc:240] Negotiation successful
0809 16:20:58.078480 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":94,"threads_started":1}
I20260809 16:20:58.079219 2779 heartbeater.cc:347] Connected to a master server at 127.21.51.190:42871
I20260809 16:20:58.079308 2779 heartbeater.cc:464] Registering TS with master...
I20260809 16:20:58.079447 2779 heartbeater.cc:511] Master 127.21.51.190:42871 requested a full tablet report, sending...
I20260809 16:20:58.079828 2005 ts_manager.cc:194] Registered new tserver with Master: 88326b268b8e485ab4d425ae71ebfc2d (127.21.51.131:36787)
I20260809 16:20:58.080156 2005 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.51.131@KRBTEST.COM'} at 127.21.51.131:45495
I20260809 16:20:58.084820 21710 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.21.51.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.21.51.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.21.51.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292417722232-21710-0/krb5kdc/oryx_127.21.51.129.keytab.
Entry for principal oryx/127.21.51.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292417722232-21710-0/krb5kdc/oryx_127.21.51.129.keytab.
WARNING: no policy specified for oryx/127.21.51.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.21.51.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.21.51.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292417722232-21710-0/krb5kdc/oryx_127.21.51.130.keytab.
Entry for principal oryx/127.21.51.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292417722232-21710-0/krb5kdc/oryx_127.21.51.130.keytab.
WARNING: no policy specified for oryx/127.21.51.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.21.51.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.21.51.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292417722232-21710-0/krb5kdc/oryx_127.21.51.131.keytab.
Entry for principal oryx/127.21.51.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292417722232-21710-0/krb5kdc/oryx_127.21.51.131.keytab.
I20260809 16:20:58.153415 21710 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrNqUCn/build/release/bin/kudu with pid 2114
I20260809 16:20:58.171461 21710 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrNqUCn/build/release/bin/kudu with pid 2352
I20260809 16:20:58.179826 21710 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrNqUCn/build/release/bin/kudu with pid 2574
I20260809 16:20:58.191274 21710 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrNqUCn/build/release/bin/kudu with pid 1950
I20260809 16:20:58.205124 21710 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrNqUCn/build/release/bin/kudu
/tmp/dist-test-taskrNqUCn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292417722232-21710-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292417722232-21710-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292417722232-21710-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292417722232-21710-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.51.190:42871
--webserver_interface=127.21.51.190
--webserver_port=46603
--keytab_file=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.190.keytab
--principal=kudu/127.21.51.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.51.148:36277
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.51.190:42871
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292417722232-21710-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292417722232-21710-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292417722232-21710-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:20:58.288451 2800 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:20:58.288681 2800 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:20:58.288750 2800 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:20:58.289880 2800 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 16:20:58.289943 2800 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:20:58.289966 2800 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:20:58.289986 2800 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 16:20:58.290016 2800 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 16:20:58.291476 2800 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.51.148:36277
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292417722232-21710-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292417722232-21710-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.51.190:42871
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.190.keytab
--principal=kudu/127.21.51.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.51.190:42871
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292417722232-21710-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.51.190
--webserver_port=46603
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2800
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292417722232-21710-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:20:58.291746 2800 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:20:58.291965 2800 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 16:20:58.296868 2800 server_base.cc:1034] running on GCE node
W20260809 16:20:58.296808 2808 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:20:58.296834 2810 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:20:58.296809 2812 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:20:58.297284 2800 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:20:58.297626 2800 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:20:58.298776 2800 hybrid_clock.cc:648] HybridClock initialized: now 1786292458298772 us; error 37 us; skew 500 ppm
Aug 09 16:20:58 dist-test-slave-s11t krb5kdc[1930](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292458, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.51.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:20:58.301764 2800 init.cc:377] Logged in from keytab as kudu/127.21.51.190@KRBTEST.COM (short username kudu)
I20260809 16:20:58.302186 2800 webserver.cc:460] Webserver started at http://127.21.51.190:46603/ using document root <none> and password file <none>
I20260809 16:20:58.302453 2800 fs_manager.cc:359] Metadata directory not provided
I20260809 16:20:58.302538 2800 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:20:58.303869 2800 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260809 16:20:58.304525 2823 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:20:58.304737 2800 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 16:20:58.304837 2800 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292417722232-21710-0/minicluster-data/master-0/data,/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292417722232-21710-0/minicluster-data/master-0/wal
uuid: "99a2f6e874df4e72ae8723a75ae18c61"
format_stamp: "Formatted at 2026-08-09 16:20:57 on dist-test-slave-s11t"
server_key: "11c36c14698a4df367d50c3a229bc163"
server_key_iv: "9241ae92f2f44a6fea576be3ae254ec7"
server_key_version: "encryptionkey@0"
I20260809 16:20:58.305167 2800 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292417722232-21710-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292417722232-21710-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292417722232-21710-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:20:58.324916 2800 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:20:58.326653 2800 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:20:58.326818 2800 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:20:58.330871 2800 rpc_server.cc:307] RPC server started. Bound to: 127.21.51.190:42871
I20260809 16:20:58.330910 2907 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.51.190:42871 every 8 connection(s)
I20260809 16:20:58.331295 2800 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292417722232-21710-0/minicluster-data/master-0/data/info.pb
I20260809 16:20:58.332794 2912 sys_catalog.cc:263] Verifying existing consensus state
I20260809 16:20:58.333477 2912 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 99a2f6e874df4e72ae8723a75ae18c61: Bootstrap starting.
I20260809 16:20:58.340378 21710 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrNqUCn/build/release/bin/kudu as pid 2800
I20260809 16:20:58.340781 21710 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrNqUCn/build/release/bin/kudu
/tmp/dist-test-taskrNqUCn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292417722232-21710-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292417722232-21710-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292417722232-21710-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292417722232-21710-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.51.129:32823
--local_ip_for_outbound_sockets=127.21.51.129
--tserver_master_addrs=127.21.51.190:42871
--webserver_port=44661
--webserver_interface=127.21.51.129
--keytab_file=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.129.keytab
--principal=kudu/127.21.51.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.51.148:36277
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.21.51.129
--keytab_file=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292417722232-21710-0/krb5kdc/oryx_127.21.51.129.keytab with env {KRB5CCNAME=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292417722232-21710-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292417722232-21710-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292417722232-21710-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 16:20:58.342121 2912 log.cc:824] T 00000000000000000000000000000000 P 99a2f6e874df4e72ae8723a75ae18c61: Log is configured to *not* fsync() on all Append() calls
I20260809 16:20:58.344089 2912 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 99a2f6e874df4e72ae8723a75ae18c61: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 16:20:58.344419 2912 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 99a2f6e874df4e72ae8723a75ae18c61: Bootstrap complete.
I20260809 16:20:58.348315 2912 raft_consensus.cc:348] T 00000000000000000000000000000000 P 99a2f6e874df4e72ae8723a75ae18c61 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "99a2f6e874df4e72ae8723a75ae18c61" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 42871 } }
I20260809 16:20:58.348724 2912 raft_consensus.cc:729] T 00000000000000000000000000000000 P 99a2f6e874df4e72ae8723a75ae18c61 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99a2f6e874df4e72ae8723a75ae18c61, State: Initialized, Role: FOLLOWER
I20260809 16:20:58.348922 2912 consensus_queue.cc:260] T 00000000000000000000000000000000 P 99a2f6e874df4e72ae8723a75ae18c61 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "99a2f6e874df4e72ae8723a75ae18c61" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 42871 } }
I20260809 16:20:58.349009 2912 raft_consensus.cc:388] T 00000000000000000000000000000000 P 99a2f6e874df4e72ae8723a75ae18c61 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 16:20:58.349069 2912 raft_consensus.cc:482] T 00000000000000000000000000000000 P 99a2f6e874df4e72ae8723a75ae18c61 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 16:20:58.349126 2912 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 99a2f6e874df4e72ae8723a75ae18c61 [term 1 FOLLOWER]: Advancing to term 2
I20260809 16:20:58.350692 2912 raft_consensus.cc:504] T 00000000000000000000000000000000 P 99a2f6e874df4e72ae8723a75ae18c61 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "99a2f6e874df4e72ae8723a75ae18c61" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 42871 } }
I20260809 16:20:58.350817 2912 leader_election.cc:302] T 00000000000000000000000000000000 P 99a2f6e874df4e72ae8723a75ae18c61 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 99a2f6e874df4e72ae8723a75ae18c61; no voters:
I20260809 16:20:58.350992 2912 leader_election.cc:288] T 00000000000000000000000000000000 P 99a2f6e874df4e72ae8723a75ae18c61 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260809 16:20:58.351058 2918 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 99a2f6e874df4e72ae8723a75ae18c61 [term 2 FOLLOWER]: Leader election won for term 2
I20260809 16:20:58.351212 2918 raft_consensus.cc:686] T 00000000000000000000000000000000 P 99a2f6e874df4e72ae8723a75ae18c61 [term 2 LEADER]: Becoming Leader. State: Replica: 99a2f6e874df4e72ae8723a75ae18c61, State: Running, Role: LEADER
I20260809 16:20:58.351293 2912 sys_catalog.cc:564] T 00000000000000000000000000000000 P 99a2f6e874df4e72ae8723a75ae18c61 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 16:20:58.351305 2918 consensus_queue.cc:237] T 00000000000000000000000000000000 P 99a2f6e874df4e72ae8723a75ae18c61 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "99a2f6e874df4e72ae8723a75ae18c61" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 42871 } }
I20260809 16:20:58.351575 2918 sys_catalog.cc:455] T 00000000000000000000000000000000 P 99a2f6e874df4e72ae8723a75ae18c61 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 99a2f6e874df4e72ae8723a75ae18c61. Latest consensus state: current_term: 2 leader_uuid: "99a2f6e874df4e72ae8723a75ae18c61" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "99a2f6e874df4e72ae8723a75ae18c61" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 42871 } } }
I20260809 16:20:58.351635 2918 sys_catalog.cc:458] T 00000000000000000000000000000000 P 99a2f6e874df4e72ae8723a75ae18c61 [sys.catalog]: This master's current role is: LEADER
I20260809 16:20:58.351794 2919 sys_catalog.cc:455] T 00000000000000000000000000000000 P 99a2f6e874df4e72ae8723a75ae18c61 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "99a2f6e874df4e72ae8723a75ae18c61" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "99a2f6e874df4e72ae8723a75ae18c61" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 42871 } } }
I20260809 16:20:58.351928 2919 sys_catalog.cc:458] T 00000000000000000000000000000000 P 99a2f6e874df4e72ae8723a75ae18c61 [sys.catalog]: This master's current role is: LEADER
I20260809 16:20:58.352550 2923 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 16:20:58.353120 2923 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 16:20:58.353284 2923 catalog_manager.cc:1259] Loaded cluster ID: 6575424e3c774195aa86544c5545bfa4
I20260809 16:20:58.353336 2923 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 16:20:58.354657 2923 catalog_manager.cc:1504] Loading token signing keys...
I20260809 16:20:58.354844 2923 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 99a2f6e874df4e72ae8723a75ae18c61: Loaded TSK: 0
I20260809 16:20:58.355048 2923 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260809 16:20:58.427232 2917 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:20:58.427426 2917 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:20:58.427449 2917 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:20:58.428586 2917 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:20:58.428627 2917 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:20:58.428653 2917 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.51.129
I20260809 16:20:58.430119 2917 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.51.148:36277
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292417722232-21710-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292417722232-21710-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292417722232-21710-0/krb5kdc/oryx_127.21.51.129.keytab
--principal=oryx/127.21.51.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.51.129:32823
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292417722232-21710-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.51.129
--webserver_port=44661
--tserver_master_addrs=127.21.51.190:42871
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2917
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.51.129
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292417722232-21710-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:20:58.430361 2917 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:20:58.430598 2917 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:20:58.435752 2959 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:20:58.435808 2963 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:20:58.435999 2917 server_base.cc:1034] running on GCE node
W20260809 16:20:58.436136 2956 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:20:58.436380 2917 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:20:58.436647 2917 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:20:58.437870 2917 hybrid_clock.cc:648] HybridClock initialized: now 1786292458437868 us; error 44 us; skew 500 ppm
Aug 09 16:20:58 dist-test-slave-s11t krb5kdc[1930](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292458, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.51.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:20:58.440769 2917 init.cc:377] Logged in from keytab as oryx/127.21.51.129@KRBTEST.COM (short username oryx)
I20260809 16:20:58.441280 2917 webserver.cc:460] Webserver started at http://127.21.51.129:44661/ using document root <none> and password file <none>
I20260809 16:20:58.441570 2917 fs_manager.cc:359] Metadata directory not provided
I20260809 16:20:58.441674 2917 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:20:58.443142 2917 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.001s
I20260809 16:20:58.443820 2977 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:20:58.444022 2917 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260809 16:20:58.444126 2917 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292417722232-21710-0/minicluster-data/ts-0/data,/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292417722232-21710-0/minicluster-data/ts-0/wal
uuid: "af2a3c461ca64de3bdd776680c9cb4ae"
format_stamp: "Formatted at 2026-08-09 16:20:57 on dist-test-slave-s11t"
server_key: "755b809679d54bbdb6078ea3896cc904"
server_key_iv: "805ad5099c30ab74d63639bdfdeded39"
server_key_version: "encryptionkey@0"
I20260809 16:20:58.444592 2917 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292417722232-21710-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292417722232-21710-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292417722232-21710-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:20:58.480579 2917 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:20:58.483404 2917 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:20:58.483556 2917 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:20:58.483815 2917 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:20:58.484155 2917 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:20:58.484197 2917 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:20:58.484226 2917 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:20:58.484242 2917 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:20:58.490546 2917 rpc_server.cc:307] RPC server started. Bound to: 127.21.51.129:32823
I20260809 16:20:58.490599 3170 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.51.129:32823 every 8 connection(s)
I20260809 16:20:58.490973 2917 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292417722232-21710-0/minicluster-data/ts-0/data/info.pb
I20260809 16:20:58.497665 21710 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrNqUCn/build/release/bin/kudu as pid 2917
Aug 09 16:20:58 dist-test-slave-s11t krb5kdc[1930](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.51.129@KRBTEST.COM for oryx/127.21.51.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 16:20:58.498006 21710 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrNqUCn/build/release/bin/kudu
/tmp/dist-test-taskrNqUCn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292417722232-21710-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292417722232-21710-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292417722232-21710-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292417722232-21710-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.51.130:46527
--local_ip_for_outbound_sockets=127.21.51.130
--tserver_master_addrs=127.21.51.190:42871
--webserver_port=39677
--webserver_interface=127.21.51.130
--keytab_file=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.130.keytab
--principal=kudu/127.21.51.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.51.148:36277
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.21.51.130
--keytab_file=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292417722232-21710-0/krb5kdc/oryx_127.21.51.130.keytab with env {KRB5CCNAME=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292417722232-21710-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292417722232-21710-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292417722232-21710-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 16:20:58 dist-test-slave-s11t krb5kdc[1930](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.51.129@KRBTEST.COM for oryx/127.21.51.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 16:20:58.499848 3175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:58.491789 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:42871 (local address 127.21.51.129:41719)
0809 16:20:58.491969 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0809 16:20:58.492012 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0809 16:20:58.492500 (+ 488us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:20:58.492986 (+ 486us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:20:58.492990 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:20:58.493697 (+ 707us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:20:58.493898 (+ 201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:20:58.493903 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:20:58.494955 (+ 1052us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:20:58.494956 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:20:58.495759 (+ 803us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:20:58.495761 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:20:58.496298 (+ 537us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:20:58.496299 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:20:58.496396 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:58.496890 (+ 494us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:20:58.496899 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:20:58.499612 (+ 2713us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.51.190:42871: Server oryx/127.21.51.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":96,"threads_started":1}
W20260809 16:20:58.500113 3171 heartbeater.cc:644] Failed to heartbeat to 127.21.51.190:42871 (0 consecutive failures): Not authorized: Failed to ping master at 127.21.51.190:42871: Client connection negotiation failed: client connection to 127.21.51.190:42871: Server oryx/127.21.51.190@KRBTEST.COM not found in Kerberos database
W20260809 16:20:58.500208 3180 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:20:58.491806 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.51.129:41719 (local address 127.21.51.190:42871)
0809 16:20:58.491981 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0809 16:20:58.491991 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 16:20:58.492159 (+ 168us) server_negotiation.cc:407] Connection header received
0809 16:20:58.492666 (+ 507us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:20:58.492671 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:20:58.492777 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:20:58.492904 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:20:58.494006 (+ 1102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:58.494866 (+ 860us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:58.495834 (+ 968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:58.496211 (+ 377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:58.496286 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:58.500029 (+ 3743us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.51.129:41719: BlockingRecv error: recv got EOF from 127.21.51.129:41719 (error 108)
Metrics: {"server-negotiator.queue_time_us":128,"thread_start_us":90,"threads_started":1}
Aug 09 16:20:58 dist-test-slave-s11t krb5kdc[1930](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.51.129@KRBTEST.COM for oryx/127.21.51.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 16:20:58 dist-test-slave-s11t krb5kdc[1930](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.51.129@KRBTEST.COM for oryx/127.21.51.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 16:20:58.504607 3175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:58.500349 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:42871 (local address 127.21.51.129:45971)
0809 16:20:58.500399 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0809 16:20:58.500412 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 16:20:58.500458 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:20:58.500602 (+ 144us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:20:58.500603 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:20:58.500782 (+ 179us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:20:58.500824 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:20:58.500826 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:20:58.501439 (+ 613us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:20:58.501441 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:20:58.501931 (+ 490us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:20:58.501933 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:20:58.502264 (+ 331us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:20:58.502265 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:20:58.502329 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:58.502743 (+ 414us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:20:58.502748 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:20:58.504536 (+ 1788us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.51.190:42871: Server oryx/127.21.51.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":32}
W20260809 16:20:58.504799 3180 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:20:58.500426 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.51.129:45971 (local address 127.21.51.190:42871)
0809 16:20:58.500463 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0809 16:20:58.500465 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:20:58.500474 (+ 9us) server_negotiation.cc:407] Connection header received
0809 16:20:58.500519 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:20:58.500520 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:20:58.500531 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:20:58.500600 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:20:58.500905 (+ 305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:58.501360 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:58.502010 (+ 650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:58.502198 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:58.502267 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:58.504711 (+ 2444us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.51.129:45971: BlockingRecv error: recv got EOF from 127.21.51.129:45971 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
Aug 09 16:20:58 dist-test-slave-s11t krb5kdc[1930](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.51.129@KRBTEST.COM for oryx/127.21.51.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 16:20:58 dist-test-slave-s11t krb5kdc[1930](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.51.129@KRBTEST.COM for oryx/127.21.51.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 16:20:58.510094 3175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:58.505367 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:42871 (local address 127.21.51.129:52817)
0809 16:20:58.505424 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0809 16:20:58.505438 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 16:20:58.505488 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:20:58.505623 (+ 135us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:20:58.505624 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:20:58.505785 (+ 161us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:20:58.505834 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:20:58.505836 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:20:58.506409 (+ 573us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:20:58.506410 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:20:58.507310 (+ 900us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:20:58.507312 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:20:58.507697 (+ 385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:20:58.507698 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:20:58.507765 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:58.508149 (+ 384us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:20:58.508155 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:20:58.510008 (+ 1853us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.51.190:42871: Server oryx/127.21.51.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":38}
W20260809 16:20:58.510123 3180 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:20:58.505432 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.51.129:52817 (local address 127.21.51.190:42871)
0809 16:20:58.505484 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0809 16:20:58.505486 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:20:58.505499 (+ 13us) server_negotiation.cc:407] Connection header received
0809 16:20:58.505545 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:20:58.505546 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:20:58.505554 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:20:58.505597 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:20:58.505915 (+ 318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:58.506336 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:58.507418 (+ 1082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:58.507625 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:58.507674 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:58.510082 (+ 2408us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.51.129:52817: BlockingRecv error: recv got EOF from 127.21.51.129:52817 (error 108)
Metrics: {"server-negotiator.queue_time_us":33}
W20260809 16:20:58.510273 3171 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260809 16:20:58.584873 3182 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:20:58.585067 3182 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:20:58.585090 3182 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:20:58.586201 3182 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:20:58.586243 3182 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:20:58.586268 3182 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.51.130
I20260809 16:20:58.587715 3182 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.51.148:36277
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292417722232-21710-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292417722232-21710-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292417722232-21710-0/krb5kdc/oryx_127.21.51.130.keytab
--principal=oryx/127.21.51.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.51.130:46527
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292417722232-21710-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.51.130
--webserver_port=39677
--tserver_master_addrs=127.21.51.190:42871
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3182
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.51.130
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292417722232-21710-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:20:58.587958 3182 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:20:58.588181 3182 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:20:58.593292 3189 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:20:58.593278 3191 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:20:58.593282 3196 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:20:58.593709 3182 server_base.cc:1034] running on GCE node
I20260809 16:20:58.593894 3182 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:20:58.594142 3182 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:20:58.595299 3182 hybrid_clock.cc:648] HybridClock initialized: now 1786292458595289 us; error 35 us; skew 500 ppm
Aug 09 16:20:58 dist-test-slave-s11t krb5kdc[1930](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292458, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.51.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:20:58.598279 3182 init.cc:377] Logged in from keytab as oryx/127.21.51.130@KRBTEST.COM (short username oryx)
I20260809 16:20:58.598737 3182 webserver.cc:460] Webserver started at http://127.21.51.130:39677/ using document root <none> and password file <none>
I20260809 16:20:58.598992 3182 fs_manager.cc:359] Metadata directory not provided
I20260809 16:20:58.599056 3182 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:20:58.600488 3182 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:20:58.601285 3208 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:20:58.601457 3182 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260809 16:20:58.601547 3182 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292417722232-21710-0/minicluster-data/ts-1/data,/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292417722232-21710-0/minicluster-data/ts-1/wal
uuid: "30adb6222bbd46ea8a9114064e90739b"
format_stamp: "Formatted at 2026-08-09 16:20:57 on dist-test-slave-s11t"
server_key: "6dc493fcd2b35a6f86d0563818bf5e49"
server_key_iv: "57fe9078a2c8e3a31940e35e5ba65f78"
server_key_version: "encryptionkey@0"
I20260809 16:20:58.601863 3182 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292417722232-21710-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292417722232-21710-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292417722232-21710-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:20:58.619511 3182 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:20:58.621711 3182 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:20:58.621863 3182 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:20:58.622067 3182 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:20:58.622394 3182 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:20:58.622430 3182 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:20:58.622454 3182 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:20:58.622502 3182 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:20:58.628430 3182 rpc_server.cc:307] RPC server started. Bound to: 127.21.51.130:46527
I20260809 16:20:58.628515 3405 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.51.130:46527 every 8 connection(s)
I20260809 16:20:58.628893 3182 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292417722232-21710-0/minicluster-data/ts-1/data/info.pb
I20260809 16:20:58.634723 21710 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrNqUCn/build/release/bin/kudu as pid 3182
I20260809 16:20:58.635041 21710 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrNqUCn/build/release/bin/kudu
/tmp/dist-test-taskrNqUCn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292417722232-21710-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292417722232-21710-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292417722232-21710-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292417722232-21710-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.51.131:36787
--local_ip_for_outbound_sockets=127.21.51.131
--tserver_master_addrs=127.21.51.190:42871
--webserver_port=46711
--webserver_interface=127.21.51.131
--keytab_file=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.131.keytab
--principal=kudu/127.21.51.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.51.148:36277
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.21.51.131
--keytab_file=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292417722232-21710-0/krb5kdc/oryx_127.21.51.131.keytab with env {KRB5CCNAME=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292417722232-21710-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292417722232-21710-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292417722232-21710-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 16:20:58 dist-test-slave-s11t krb5kdc[1930](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.51.130@KRBTEST.COM for oryx/127.21.51.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 16:20:58 dist-test-slave-s11t krb5kdc[1930](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.51.130@KRBTEST.COM for oryx/127.21.51.190@KRBTEST.COM, Server not found in Kerberos database
W20260809 16:20:58.636957 3180 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:20:58.629649 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.51.130:55925 (local address 127.21.51.190:42871)
0809 16:20:58.629764 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0809 16:20:58.629767 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 16:20:58.630003 (+ 236us) server_negotiation.cc:407] Connection header received
0809 16:20:58.630503 (+ 500us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:20:58.630504 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:20:58.630518 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:20:58.630576 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:20:58.631597 (+ 1021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:58.632130 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:58.633249 (+ 1119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:58.633464 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:58.633536 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:58.636910 (+ 3374us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.51.130:55925: BlockingRecv error: recv got EOF from 127.21.51.130:55925 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
I20260809 16:20:58.637039 3410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:58.629624 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:42871 (local address 127.21.51.130:55925)
0809 16:20:58.629869 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0809 16:20:58.629916 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0809 16:20:58.630415 (+ 499us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:20:58.630660 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:20:58.630668 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:20:58.631290 (+ 622us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:20:58.631508 (+ 218us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:20:58.631513 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:20:58.632220 (+ 707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:20:58.632220 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:20:58.633130 (+ 910us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:20:58.633133 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:20:58.633550 (+ 417us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:20:58.633550 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:20:58.633652 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:58.634179 (+ 527us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:20:58.634188 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:20:58.636798 (+ 2610us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.51.190:42871: Server oryx/127.21.51.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":105,"threads_started":1}
W20260809 16:20:58.637277 3408 heartbeater.cc:644] Failed to heartbeat to 127.21.51.190:42871 (0 consecutive failures): Not authorized: Failed to ping master at 127.21.51.190:42871: Client connection negotiation failed: client connection to 127.21.51.190:42871: Server oryx/127.21.51.190@KRBTEST.COM not found in Kerberos database
Aug 09 16:20:58 dist-test-slave-s11t krb5kdc[1930](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.51.130@KRBTEST.COM for oryx/127.21.51.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 16:20:58 dist-test-slave-s11t krb5kdc[1930](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.51.130@KRBTEST.COM for oryx/127.21.51.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 16:20:58.642511 3410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:58.637513 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:42871 (local address 127.21.51.130:37403)
0809 16:20:58.637558 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0809 16:20:58.637569 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 16:20:58.637630 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:20:58.637759 (+ 129us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:20:58.637760 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:20:58.637922 (+ 162us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:20:58.637976 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:20:58.637978 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:20:58.638612 (+ 634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:20:58.638614 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:20:58.639497 (+ 883us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:20:58.639500 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:20:58.639930 (+ 430us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:20:58.639931 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:20:58.640014 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:58.640415 (+ 401us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:20:58.640421 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:20:58.642412 (+ 1991us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.51.190:42871: Server oryx/127.21.51.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":29}
W20260809 16:20:58.642536 3180 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:20:58.637567 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.51.130:37403 (local address 127.21.51.190:42871)
0809 16:20:58.637605 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0809 16:20:58.637607 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:20:58.637616 (+ 9us) server_negotiation.cc:407] Connection header received
0809 16:20:58.637687 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:20:58.637688 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:20:58.637697 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:20:58.637757 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:20:58.638054 (+ 297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:58.638529 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:58.639611 (+ 1082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:58.639851 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:58.639914 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:58.642491 (+ 2577us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.51.130:37403: BlockingRecv error: recv got EOF from 127.21.51.130:37403 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
Aug 09 16:20:58 dist-test-slave-s11t krb5kdc[1930](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.51.130@KRBTEST.COM for oryx/127.21.51.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 16:20:58 dist-test-slave-s11t krb5kdc[1930](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.51.130@KRBTEST.COM for oryx/127.21.51.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 16:20:58.647446 3410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:58.642801 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:42871 (local address 127.21.51.130:33793)
0809 16:20:58.642926 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0809 16:20:58.642937 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 16:20:58.642995 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:20:58.643131 (+ 136us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:20:58.643132 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:20:58.643301 (+ 169us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:20:58.643337 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:20:58.643338 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:20:58.643919 (+ 581us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:20:58.643920 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:20:58.644832 (+ 912us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:20:58.644835 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:20:58.645165 (+ 330us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:20:58.645166 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:20:58.645241 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:58.645602 (+ 361us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:20:58.645608 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:20:58.647366 (+ 1758us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.51.190:42871: Server oryx/127.21.51.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":106}
W20260809 16:20:58.647504 3180 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:20:58.642876 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.51.130:33793 (local address 127.21.51.190:42871)
0809 16:20:58.642933 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0809 16:20:58.642935 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:20:58.642979 (+ 44us) server_negotiation.cc:407] Connection header received
0809 16:20:58.643056 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:20:58.643057 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:20:58.643064 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:20:58.643109 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:20:58.643412 (+ 303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:58.643845 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:58.644919 (+ 1074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:58.645091 (+ 172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:58.645144 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:58.647443 (+ 2299us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.51.130:33793: BlockingRecv error: recv got EOF from 127.21.51.130:33793 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
W20260809 16:20:58.647612 3408 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260809 16:20:58.721742 3411 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:20:58.721930 3411 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:20:58.721953 3411 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:20:58.723080 3411 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:20:58.723124 3411 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:20:58.723150 3411 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.51.131
I20260809 16:20:58.724547 3411 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.51.148:36277
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292417722232-21710-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292417722232-21710-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292417722232-21710-0/krb5kdc/oryx_127.21.51.131.keytab
--principal=oryx/127.21.51.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.51.131:36787
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292417722232-21710-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.51.131
--webserver_port=46711
--tserver_master_addrs=127.21.51.190:42871
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3411
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.51.131
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292417722232-21710-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:20:58.724820 3411 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:20:58.725051 3411 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:20:58.730093 3417 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:20:58.730088 3421 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:20:58.730229 3418 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:20:58.730598 3411 server_base.cc:1034] running on GCE node
I20260809 16:20:58.730763 3411 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:20:58.730997 3411 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:20:58.732164 3411 hybrid_clock.cc:648] HybridClock initialized: now 1786292458732148 us; error 43 us; skew 500 ppm
Aug 09 16:20:58 dist-test-slave-s11t krb5kdc[1930](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292458, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.51.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:20:58.734696 3411 init.cc:377] Logged in from keytab as oryx/127.21.51.131@KRBTEST.COM (short username oryx)
I20260809 16:20:58.735069 3411 webserver.cc:460] Webserver started at http://127.21.51.131:46711/ using document root <none> and password file <none>
I20260809 16:20:58.735268 3411 fs_manager.cc:359] Metadata directory not provided
I20260809 16:20:58.735320 3411 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:20:58.736596 3411 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:20:58.737372 3433 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:20:58.737550 3411 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260809 16:20:58.737661 3411 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292417722232-21710-0/minicluster-data/ts-2/data,/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292417722232-21710-0/minicluster-data/ts-2/wal
uuid: "88326b268b8e485ab4d425ae71ebfc2d"
format_stamp: "Formatted at 2026-08-09 16:20:58 on dist-test-slave-s11t"
server_key: "5c0561e84b6f92beb5e7db9ee107b109"
server_key_iv: "1ca708d63c0980734722361b9f51a026"
server_key_version: "encryptionkey@0"
I20260809 16:20:58.737980 3411 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292417722232-21710-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292417722232-21710-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292417722232-21710-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:20:58.754807 3411 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:20:58.757141 3411 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:20:58.757308 3411 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:20:58.757540 3411 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:20:58.757848 3411 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:20:58.757881 3411 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:20:58.757931 3411 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:20:58.757961 3411 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:20:58.763801 3411 rpc_server.cc:307] RPC server started. Bound to: 127.21.51.131:36787
I20260809 16:20:58.763845 3605 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.51.131:36787 every 8 connection(s)
I20260809 16:20:58.764195 3411 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292417722232-21710-0/minicluster-data/ts-2/data/info.pb
Aug 09 16:20:58 dist-test-slave-s11t krb5kdc[1930](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.51.131@KRBTEST.COM for oryx/127.21.51.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 16:20:58 dist-test-slave-s11t krb5kdc[1930](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.51.131@KRBTEST.COM for oryx/127.21.51.190@KRBTEST.COM, Server not found in Kerberos database
W20260809 16:20:58.771518 3180 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:20:58.765054 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.51.131:49765 (local address 127.21.51.190:42871)
0809 16:20:58.765142 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0809 16:20:58.765144 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:20:58.765319 (+ 175us) server_negotiation.cc:407] Connection header received
0809 16:20:58.765817 (+ 498us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:20:58.765818 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:20:58.765829 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:20:58.765894 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:20:58.766805 (+ 911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:58.767320 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:58.768310 (+ 990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:58.768501 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:58.768560 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:58.771471 (+ 2911us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.51.131:49765: BlockingRecv error: recv got EOF from 127.21.51.131:49765 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
I20260809 16:20:58.771545 21710 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrNqUCn/build/release/bin/kudu as pid 3411
I20260809 16:20:58.771612 3609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:58.764985 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:42871 (local address 127.21.51.131:49765)
0809 16:20:58.765158 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0809 16:20:58.765200 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0809 16:20:58.765682 (+ 482us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:20:58.765946 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:20:58.765950 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:20:58.766505 (+ 555us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:20:58.766695 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:20:58.766700 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:20:58.767422 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:20:58.767423 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:20:58.768228 (+ 805us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:20:58.768230 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:20:58.768577 (+ 347us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:20:58.768577 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:20:58.768674 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:58.769198 (+ 524us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:20:58.769205 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:20:58.771391 (+ 2186us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.51.190:42871: Server oryx/127.21.51.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":87,"threads_started":1}
W20260809 16:20:58.771904 3606 heartbeater.cc:644] Failed to heartbeat to 127.21.51.190:42871 (0 consecutive failures): Not authorized: Failed to ping master at 127.21.51.190:42871: Client connection negotiation failed: client connection to 127.21.51.190:42871: Server oryx/127.21.51.190@KRBTEST.COM not found in Kerberos database
Aug 09 16:20:58 dist-test-slave-s11t krb5kdc[1930](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.51.131@KRBTEST.COM for oryx/127.21.51.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 16:20:58 dist-test-slave-s11t krb5kdc[1930](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.51.131@KRBTEST.COM for oryx/127.21.51.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 16:20:58.776731 3609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:58.772145 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:42871 (local address 127.21.51.131:33889)
0809 16:20:58.772196 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0809 16:20:58.772207 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 16:20:58.772255 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:20:58.772443 (+ 188us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:20:58.772444 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:20:58.772570 (+ 126us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:20:58.772612 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:20:58.772613 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:20:58.773315 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:20:58.773316 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:20:58.773802 (+ 486us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:20:58.773804 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:20:58.774168 (+ 364us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:20:58.774169 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:20:58.774241 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:58.774629 (+ 388us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:20:58.774638 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:20:58.776632 (+ 1994us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.51.190:42871: Server oryx/127.21.51.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":34}
W20260809 16:20:58.776851 3611 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:20:58.772182 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.51.131:33889 (local address 127.21.51.190:42871)
0809 16:20:58.772320 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0809 16:20:58.772330 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 16:20:58.772336 (+ 6us) server_negotiation.cc:407] Connection header received
0809 16:20:58.772360 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:20:58.772360 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:20:58.772368 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:20:58.772438 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:20:58.772761 (+ 323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:58.773236 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:58.773882 (+ 646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:58.774098 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:58.774170 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:58.776778 (+ 2608us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.51.131:33889: BlockingRecv error: recv got EOF from 127.21.51.131:33889 (error 108)
Metrics: {"server-negotiator.queue_time_us":121,"thread_start_us":82,"threads_started":1}
I20260809 16:20:58.779284 3180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:58.772054 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41648 (local address 127.21.51.190:42871)
0809 16:20:58.772102 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0809 16:20:58.772104 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:20:58.772171 (+ 67us) server_negotiation.cc:407] Connection header received
0809 16:20:58.772280 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:20:58.772281 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:20:58.772290 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:20:58.772348 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:20:58.773007 (+ 659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:58.773460 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:58.774208 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:58.774401 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:58.774471 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:58.775831 (+ 1360us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:20:58.775836 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:20:58.775836 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:20:58.775848 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:20:58.777921 (+ 2073us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:20:58.778268 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:20:58.778272 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:20:58.778272 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:20:58.778365 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:20:58.778577 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:20:58.778578 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:20:58.778578 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:20:58.778800 (+ 222us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:20:58.778966 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0809 16:20:58.779063 (+ 97us) server_negotiation.cc:299] Negotiation successful
0809 16:20:58.779172 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
Aug 09 16:20:58 dist-test-slave-s11t krb5kdc[1930](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.51.131@KRBTEST.COM for oryx/127.21.51.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 16:20:58 dist-test-slave-s11t krb5kdc[1930](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.51.131@KRBTEST.COM for oryx/127.21.51.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 16:20:58.781459 3609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:58.777244 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:42871 (local address 127.21.51.131:56237)
0809 16:20:58.777297 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0809 16:20:58.777308 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 16:20:58.777354 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:20:58.777476 (+ 122us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:20:58.777477 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:20:58.777605 (+ 128us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:20:58.777637 (+ 32us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:20:58.777638 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:20:58.778258 (+ 620us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:20:58.778259 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:20:58.778734 (+ 475us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:20:58.778735 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:20:58.779084 (+ 349us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:20:58.779085 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:20:58.779138 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:58.779540 (+ 402us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:20:58.779545 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:20:58.781419 (+ 1874us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.51.190:42871: Server oryx/127.21.51.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":37}
W20260809 16:20:58.781598 3606 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260809 16:20:58.781690 3611 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:20:58.777293 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.51.131:56237 (local address 127.21.51.190:42871)
0809 16:20:58.777338 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0809 16:20:58.777340 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:20:58.777348 (+ 8us) server_negotiation.cc:407] Connection header received
0809 16:20:58.777420 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:20:58.777420 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:20:58.777429 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:20:58.777471 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:20:58.777714 (+ 243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:58.778176 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:58.778811 (+ 635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:58.779004 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:58.779083 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:58.781612 (+ 2529us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.51.131:56237: BlockingRecv error: recv got EOF from 127.21.51.131:56237 (error 108)
Metrics: {"server-negotiator.queue_time_us":28}
Aug 09 16:20:59 dist-test-slave-s11t krb5kdc[1930](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.51.129@KRBTEST.COM for oryx/127.21.51.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 16:20:59 dist-test-slave-s11t krb5kdc[1930](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.51.129@KRBTEST.COM for oryx/127.21.51.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 16:20:59.515640 3613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:59.510649 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:42871 (local address 127.21.51.129:55155)
0809 16:20:59.510838 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0809 16:20:59.510851 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 16:20:59.510919 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:20:59.511061 (+ 142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:20:59.511062 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:20:59.511238 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:20:59.511321 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:20:59.511323 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:20:59.511980 (+ 657us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:20:59.511981 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:20:59.512529 (+ 548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:20:59.512530 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:20:59.512923 (+ 393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:20:59.512925 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:20:59.512976 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:59.513404 (+ 428us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:20:59.513409 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:20:59.515535 (+ 2126us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.51.190:42871: Server oryx/127.21.51.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":89,"threads_started":1}
W20260809 16:20:59.515741 3614 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:20:59.510738 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.51.129:55155 (local address 127.21.51.190:42871)
0809 16:20:59.510859 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0809 16:20:59.510861 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:20:59.510902 (+ 41us) server_negotiation.cc:407] Connection header received
0809 16:20:59.510971 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:20:59.510972 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:20:59.510982 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:20:59.511041 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:20:59.511404 (+ 363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:59.511909 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:59.512601 (+ 692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:59.512835 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:59.512893 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:59.515646 (+ 2753us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.51.129:55155: BlockingRecv error: recv got EOF from 127.21.51.129:55155 (error 108)
Metrics: {"server-negotiator.queue_time_us":103,"thread_start_us":70,"threads_started":1}
Aug 09 16:20:59 dist-test-slave-s11t krb5kdc[1930](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.51.130@KRBTEST.COM for oryx/127.21.51.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 16:20:59 dist-test-slave-s11t krb5kdc[1930](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.51.130@KRBTEST.COM for oryx/127.21.51.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 16:20:59.653208 3615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:59.648041 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:42871 (local address 127.21.51.130:46251)
0809 16:20:59.648189 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0809 16:20:59.648201 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 16:20:59.648265 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:20:59.648426 (+ 161us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:20:59.648428 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:20:59.648622 (+ 194us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:20:59.648671 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:20:59.648672 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:20:59.649485 (+ 813us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:20:59.649486 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:20:59.650033 (+ 547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:20:59.650035 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:20:59.650419 (+ 384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:20:59.650420 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:20:59.650479 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:59.650995 (+ 516us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:20:59.651001 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:20:59.653089 (+ 2088us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.51.190:42871: Server oryx/127.21.51.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":100,"threads_started":1}
W20260809 16:20:59.653232 3614 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:20:59.648109 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.51.130:46251 (local address 127.21.51.190:42871)
0809 16:20:59.648235 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0809 16:20:59.648237 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:20:59.648245 (+ 8us) server_negotiation.cc:407] Connection header received
0809 16:20:59.648338 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:20:59.648339 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:20:59.648352 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:20:59.648423 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:20:59.648790 (+ 367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:59.649403 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:59.650128 (+ 725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:59.650337 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:59.650417 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:59.653170 (+ 2753us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.51.130:46251: BlockingRecv error: recv got EOF from 127.21.51.130:46251 (error 108)
Metrics: {"server-negotiator.queue_time_us":97}
Aug 09 16:20:59 dist-test-slave-s11t krb5kdc[1930](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.51.131@KRBTEST.COM for oryx/127.21.51.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 16:20:59 dist-test-slave-s11t krb5kdc[1930](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.51.131@KRBTEST.COM for oryx/127.21.51.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 16:20:59.786599 3616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:20:59.781946 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:42871 (local address 127.21.51.131:35343)
0809 16:20:59.782061 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0809 16:20:59.782071 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 16:20:59.782124 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:20:59.782250 (+ 126us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:20:59.782250 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:20:59.782411 (+ 161us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:20:59.782461 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:20:59.782462 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:20:59.783093 (+ 631us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:20:59.783093 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:20:59.783649 (+ 556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:20:59.783651 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:20:59.784202 (+ 551us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:20:59.784203 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:20:59.784264 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:59.784741 (+ 477us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:20:59.784746 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:20:59.786536 (+ 1790us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.51.190:42871: Server oryx/127.21.51.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":88,"thread_start_us":52,"threads_started":1}
W20260809 16:20:59.786798 3614 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:20:59.782022 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.51.131:35343 (local address 127.21.51.190:42871)
0809 16:20:59.782107 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0809 16:20:59.782109 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:20:59.782119 (+ 10us) server_negotiation.cc:407] Connection header received
0809 16:20:59.782169 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:20:59.782169 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:20:59.782184 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:20:59.782234 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:20:59.782522 (+ 288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:59.783029 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:59.783788 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:20:59.784109 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:20:59.784179 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:20:59.786692 (+ 2513us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.51.131:35343: BlockingRecv error: recv got EOF from 127.21.51.131:35343 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
Aug 09 16:21:00 dist-test-slave-s11t krb5kdc[1930](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.51.129@KRBTEST.COM for oryx/127.21.51.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 16:21:00 dist-test-slave-s11t krb5kdc[1930](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.51.129@KRBTEST.COM for oryx/127.21.51.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 16:21:00.520964 3621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:21:00.516158 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:42871 (local address 127.21.51.129:40931)
0809 16:21:00.516352 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0809 16:21:00.516365 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 16:21:00.516432 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:21:00.516595 (+ 163us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:21:00.516596 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:21:00.516791 (+ 195us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:21:00.516840 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:21:00.516842 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:21:00.517546 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:21:00.517547 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:21:00.518077 (+ 530us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:21:00.518078 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:21:00.518445 (+ 367us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:21:00.518446 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:21:00.518507 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:21:00.519009 (+ 502us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:21:00.519014 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:21:00.520852 (+ 1838us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.51.190:42871: Server oryx/127.21.51.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":66,"threads_started":1}
W20260809 16:21:00.521008 3624 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:21:00.516217 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.51.129:40931 (local address 127.21.51.190:42871)
0809 16:21:00.516328 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0809 16:21:00.516330 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:21:00.516438 (+ 108us) server_negotiation.cc:407] Connection header received
0809 16:21:00.516486 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:21:00.516487 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:21:00.516502 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:21:00.516582 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:21:00.516926 (+ 344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:21:00.517453 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:21:00.518159 (+ 706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:21:00.518363 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:21:00.518431 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:21:00.520936 (+ 2505us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.51.129:40931: BlockingRecv error: recv got EOF from 127.21.51.129:40931 (error 108)
Metrics: {"server-negotiator.queue_time_us":86,"thread_start_us":48,"threads_started":1}
Aug 09 16:21:00 dist-test-slave-s11t krb5kdc[1930](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.51.130@KRBTEST.COM for oryx/127.21.51.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 16:21:00 dist-test-slave-s11t krb5kdc[1930](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.51.130@KRBTEST.COM for oryx/127.21.51.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 16:21:00.658586 3626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:21:00.653837 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:42871 (local address 127.21.51.130:58493)
0809 16:21:00.654016 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0809 16:21:00.654026 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 16:21:00.654086 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:21:00.654231 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:21:00.654232 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:21:00.654408 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:21:00.654460 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:21:00.654462 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:21:00.655101 (+ 639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:21:00.655102 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:21:00.655653 (+ 551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:21:00.655654 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:21:00.656025 (+ 371us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:21:00.656026 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:21:00.656067 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:21:00.656605 (+ 538us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:21:00.656611 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:21:00.658489 (+ 1878us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.51.190:42871: Server oryx/127.21.51.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":121,"threads_started":1}
W20260809 16:21:00.658636 3624 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:21:00.653902 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.51.130:58493 (local address 127.21.51.190:42871)
0809 16:21:00.653975 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0809 16:21:00.653977 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:21:00.654105 (+ 128us) server_negotiation.cc:407] Connection header received
0809 16:21:00.654137 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:21:00.654138 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:21:00.654151 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:21:00.654226 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:21:00.654521 (+ 295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:21:00.655032 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:21:00.655736 (+ 704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:21:00.655956 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:21:00.656017 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:21:00.658572 (+ 2555us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.51.130:58493: BlockingRecv error: recv got EOF from 127.21.51.130:58493 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
Aug 09 16:21:00 dist-test-slave-s11t krb5kdc[1930](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.51.131@KRBTEST.COM for oryx/127.21.51.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 16:21:00 dist-test-slave-s11t krb5kdc[1930](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.51.131@KRBTEST.COM for oryx/127.21.51.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 16:21:00.791798 3628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:21:00.787064 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:42871 (local address 127.21.51.131:57125)
0809 16:21:00.787216 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0809 16:21:00.787238 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0809 16:21:00.787286 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:21:00.787409 (+ 123us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:21:00.787409 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:21:00.787578 (+ 169us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:21:00.787629 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:21:00.787630 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:21:00.788421 (+ 791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:21:00.788422 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:21:00.789040 (+ 618us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:21:00.789042 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:21:00.789413 (+ 371us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:21:00.789413 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:21:00.789460 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:21:00.790001 (+ 541us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:21:00.790006 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:21:00.791739 (+ 1733us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.51.190:42871: Server oryx/127.21.51.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":100,"threads_started":1}
W20260809 16:21:00.791930 3624 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:21:00.787185 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.51.131:57125 (local address 127.21.51.190:42871)
0809 16:21:00.787267 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0809 16:21:00.787269 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:21:00.787280 (+ 11us) server_negotiation.cc:407] Connection header received
0809 16:21:00.787337 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:21:00.787338 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:21:00.787351 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:21:00.787402 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:21:00.787690 (+ 288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:21:00.788242 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:21:00.789134 (+ 892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:21:00.789327 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:21:00.789385 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:21:00.791865 (+ 2480us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.51.131:57125: BlockingRecv error: recv got EOF from 127.21.51.131:57125 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
Aug 09 16:21:01 dist-test-slave-s11t krb5kdc[1930](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.51.129@KRBTEST.COM for oryx/127.21.51.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 16:21:01 dist-test-slave-s11t krb5kdc[1930](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.51.129@KRBTEST.COM for oryx/127.21.51.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 16:21:01.526134 3630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:21:01.521453 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:42871 (local address 127.21.51.129:56823)
0809 16:21:01.521603 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0809 16:21:01.521615 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 16:21:01.521668 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:21:01.521818 (+ 150us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:21:01.521818 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:21:01.521988 (+ 170us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:21:01.522041 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:21:01.522043 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:21:01.522761 (+ 718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:21:01.522761 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:21:01.523295 (+ 534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:21:01.523297 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:21:01.523664 (+ 367us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:21:01.523665 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:21:01.523707 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:21:01.524156 (+ 449us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:21:01.524161 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:21:01.526034 (+ 1873us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.51.190:42871: Server oryx/127.21.51.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":85,"threads_started":1}
W20260809 16:21:01.526206 3632 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:21:01.521531 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.51.129:56823 (local address 127.21.51.190:42871)
0809 16:21:01.521640 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0809 16:21:01.521642 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:21:01.521656 (+ 14us) server_negotiation.cc:407] Connection header received
0809 16:21:01.521730 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:21:01.521731 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:21:01.521742 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:21:01.521805 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:21:01.522132 (+ 327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:21:01.522645 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:21:01.523390 (+ 745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:21:01.523592 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:21:01.523658 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:21:01.526134 (+ 2476us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.51.129:56823: BlockingRecv error: recv got EOF from 127.21.51.129:56823 (error 108)
Metrics: {"server-negotiator.queue_time_us":84,"thread_start_us":55,"threads_started":1}
Aug 09 16:21:01 dist-test-slave-s11t krb5kdc[1930](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.51.130@KRBTEST.COM for oryx/127.21.51.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 16:21:01 dist-test-slave-s11t krb5kdc[1930](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.51.130@KRBTEST.COM for oryx/127.21.51.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 16:21:01.663806 3635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:21:01.659149 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:42871 (local address 127.21.51.130:42013)
0809 16:21:01.659324 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0809 16:21:01.659336 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 16:21:01.659399 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:21:01.659528 (+ 129us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:21:01.659529 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:21:01.659697 (+ 168us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:21:01.659763 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:21:01.659764 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:21:01.660416 (+ 652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:21:01.660417 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:21:01.661120 (+ 703us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:21:01.661124 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:21:01.661462 (+ 338us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:21:01.661463 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:21:01.661505 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:21:01.661945 (+ 440us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:21:01.661950 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:21:01.663736 (+ 1786us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.51.190:42871: Server oryx/127.21.51.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":126,"threads_started":1}
W20260809 16:21:01.663897 3632 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:21:01.659220 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.51.130:42013 (local address 127.21.51.190:42871)
0809 16:21:01.659296 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0809 16:21:01.659298 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:21:01.659411 (+ 113us) server_negotiation.cc:407] Connection header received
0809 16:21:01.659444 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:21:01.659445 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:21:01.659457 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:21:01.659515 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:21:01.659825 (+ 310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:21:01.660338 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:21:01.661205 (+ 867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:21:01.661398 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:21:01.661454 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:21:01.663830 (+ 2376us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.51.130:42013: BlockingRecv error: recv got EOF from 127.21.51.130:42013 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
Aug 09 16:21:01 dist-test-slave-s11t krb5kdc[1930](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.51.131@KRBTEST.COM for oryx/127.21.51.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 16:21:01 dist-test-slave-s11t krb5kdc[1930](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.51.131@KRBTEST.COM for oryx/127.21.51.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 16:21:01.797451 3636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:21:01.792484 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:42871 (local address 127.21.51.131:41123)
0809 16:21:01.792674 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0809 16:21:01.792706 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0809 16:21:01.792774 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:21:01.792950 (+ 176us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:21:01.792951 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:21:01.793160 (+ 209us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:21:01.793212 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:21:01.793214 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:21:01.793854 (+ 640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:21:01.793856 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:21:01.794462 (+ 606us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:21:01.794464 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:21:01.794783 (+ 319us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:21:01.794784 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:21:01.794828 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:21:01.795276 (+ 448us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:21:01.795281 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:21:01.797329 (+ 2048us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.51.190:42871: Server oryx/127.21.51.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":104,"threads_started":1}
W20260809 16:21:01.797461 3632 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:21:01.792559 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.51.131:41123 (local address 127.21.51.190:42871)
0809 16:21:01.792633 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0809 16:21:01.792635 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:21:01.792769 (+ 134us) server_negotiation.cc:407] Connection header received
0809 16:21:01.792824 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:21:01.792824 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:21:01.792838 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:21:01.792909 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:21:01.793278 (+ 369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:21:01.793773 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:21:01.794540 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:21:01.794719 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:21:01.794770 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:21:01.797405 (+ 2635us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.51.131:41123: BlockingRecv error: recv got EOF from 127.21.51.131:41123 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
Aug 09 16:21:02 dist-test-slave-s11t krb5kdc[1930](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.51.129@KRBTEST.COM for oryx/127.21.51.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 16:21:02 dist-test-slave-s11t krb5kdc[1930](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.51.129@KRBTEST.COM for oryx/127.21.51.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 16:21:02.535981 3637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:21:02.526623 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:42871 (local address 127.21.51.129:39207)
0809 16:21:02.526772 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0809 16:21:02.526782 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 16:21:02.526830 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:21:02.527065 (+ 235us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:21:02.527066 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:21:02.527245 (+ 179us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:21:02.527299 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:21:02.527301 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:21:02.528019 (+ 718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:21:02.528020 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:21:02.528567 (+ 547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:21:02.528568 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:21:02.529022 (+ 454us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:21:02.529022 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:21:02.529071 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:21:02.534080 (+ 5009us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:21:02.534085 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:21:02.535920 (+ 1835us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.51.190:42871: Server oryx/127.21.51.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":83,"threads_started":1}
W20260809 16:21:02.536139 3639 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:21:02.526758 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.51.129:39207 (local address 127.21.51.190:42871)
0809 16:21:02.526937 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0809 16:21:02.526939 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:21:02.526948 (+ 9us) server_negotiation.cc:407] Connection header received
0809 16:21:02.526970 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:21:02.526971 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:21:02.526981 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:21:02.527050 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:21:02.527377 (+ 327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:21:02.527942 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:21:02.528665 (+ 723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:21:02.528941 (+ 276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:21:02.528999 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:21:02.536047 (+ 7048us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.51.129:39207: BlockingRecv error: recv got EOF from 127.21.51.129:39207 (error 108)
Metrics: {"server-negotiator.queue_time_us":153,"thread_start_us":119,"threads_started":1}
Aug 09 16:21:02 dist-test-slave-s11t krb5kdc[1930](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.51.130@KRBTEST.COM for oryx/127.21.51.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 16:21:02 dist-test-slave-s11t krb5kdc[1930](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.51.130@KRBTEST.COM for oryx/127.21.51.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 16:21:02.669096 3641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:21:02.664381 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:42871 (local address 127.21.51.130:45447)
0809 16:21:02.664531 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0809 16:21:02.664543 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 16:21:02.664613 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:21:02.664767 (+ 154us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:21:02.664768 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:21:02.664965 (+ 197us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:21:02.665014 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:21:02.665016 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:21:02.665713 (+ 697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:21:02.665715 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:21:02.666238 (+ 523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:21:02.666240 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:21:02.666613 (+ 373us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:21:02.666614 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:21:02.666675 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:21:02.667159 (+ 484us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:21:02.667164 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:21:02.668998 (+ 1834us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.51.190:42871: Server oryx/127.21.51.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":88,"threads_started":1}
W20260809 16:21:02.669149 3639 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:21:02.664470 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.51.130:45447 (local address 127.21.51.190:42871)
0809 16:21:02.664528 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0809 16:21:02.664530 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:21:02.664595 (+ 65us) server_negotiation.cc:407] Connection header received
0809 16:21:02.664669 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:21:02.664670 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:21:02.664698 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:21:02.664749 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:21:02.665099 (+ 350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:21:02.665622 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:21:02.666320 (+ 698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:21:02.666528 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:21:02.666612 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:21:02.669084 (+ 2472us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.51.130:45447: BlockingRecv error: recv got EOF from 127.21.51.130:45447 (error 108)
Metrics: {"server-negotiator.queue_time_us":31}
Aug 09 16:21:02 dist-test-slave-s11t krb5kdc[1930](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.51.131@KRBTEST.COM for oryx/127.21.51.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 16:21:02 dist-test-slave-s11t krb5kdc[1930](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.51.131@KRBTEST.COM for oryx/127.21.51.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 16:21:02.802318 3643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:21:02.797885 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:42871 (local address 127.21.51.131:59399)
0809 16:21:02.798040 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0809 16:21:02.798052 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 16:21:02.798115 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:21:02.798245 (+ 130us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:21:02.798246 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:21:02.798413 (+ 167us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:21:02.798465 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:21:02.798466 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:21:02.799094 (+ 628us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:21:02.799095 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:21:02.799620 (+ 525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:21:02.799622 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:21:02.799942 (+ 320us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:21:02.799943 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:21:02.799985 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:21:02.800485 (+ 500us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:21:02.800490 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:21:02.802218 (+ 1728us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.51.190:42871: Server oryx/127.21.51.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":68,"threads_started":1}
W20260809 16:21:02.802359 3639 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:21:02.797953 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.51.131:59399 (local address 127.21.51.190:42871)
0809 16:21:02.798039 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0809 16:21:02.798041 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:21:02.798097 (+ 56us) server_negotiation.cc:407] Connection header received
0809 16:21:02.798157 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:21:02.798158 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:21:02.798171 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:21:02.798226 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:21:02.798523 (+ 297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:21:02.799024 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:21:02.799687 (+ 663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:21:02.799863 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:21:02.799930 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:21:02.802297 (+ 2367us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.51.131:59399: BlockingRecv error: recv got EOF from 127.21.51.131:59399 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
Aug 09 16:21:03 dist-test-slave-s11t krb5kdc[1930](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.51.129@KRBTEST.COM for oryx/127.21.51.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 16:21:03 dist-test-slave-s11t krb5kdc[1930](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.51.129@KRBTEST.COM for oryx/127.21.51.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 16:21:03.541533 3644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:21:03.536497 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:42871 (local address 127.21.51.129:57653)
0809 16:21:03.536638 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0809 16:21:03.536652 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 16:21:03.536751 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:21:03.536959 (+ 208us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:21:03.536960 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:21:03.537131 (+ 171us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:21:03.537189 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:21:03.537190 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:21:03.538072 (+ 882us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:21:03.538073 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:21:03.538628 (+ 555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:21:03.538631 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:21:03.539030 (+ 399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:21:03.539031 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:21:03.539090 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:21:03.539550 (+ 460us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:21:03.539555 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:21:03.541472 (+ 1917us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.51.190:42871: Server oryx/127.21.51.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":85,"threads_started":1}
W20260809 16:21:03.541679 3645 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:21:03.536603 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.51.129:57653 (local address 127.21.51.190:42871)
0809 16:21:03.536792 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0809 16:21:03.536794 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:21:03.536803 (+ 9us) server_negotiation.cc:407] Connection header received
0809 16:21:03.536861 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:21:03.536861 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:21:03.536872 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:21:03.536942 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:21:03.537266 (+ 324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:21:03.538001 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:21:03.538732 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:21:03.538948 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:21:03.539015 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:21:03.541566 (+ 2551us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.51.129:57653: BlockingRecv error: recv got EOF from 127.21.51.129:57653 (error 108)
Metrics: {"server-negotiator.queue_time_us":162,"thread_start_us":134,"threads_started":1}
Aug 09 16:21:03 dist-test-slave-s11t krb5kdc[1930](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.51.130@KRBTEST.COM for oryx/127.21.51.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 16:21:03 dist-test-slave-s11t krb5kdc[1930](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.51.130@KRBTEST.COM for oryx/127.21.51.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 16:21:03.674464 3646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:21:03.669650 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:42871 (local address 127.21.51.130:56533)
0809 16:21:03.669803 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0809 16:21:03.669817 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 16:21:03.669888 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:21:03.670057 (+ 169us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:21:03.670058 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:21:03.670243 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:21:03.670306 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:21:03.670307 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:21:03.671006 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:21:03.671007 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:21:03.671566 (+ 559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:21:03.671567 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:21:03.671934 (+ 367us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:21:03.671935 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:21:03.671978 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:21:03.672450 (+ 472us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:21:03.672455 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:21:03.674404 (+ 1949us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.51.190:42871: Server oryx/127.21.51.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":165,"threads_started":1}
W20260809 16:21:03.674607 3645 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:21:03.669852 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.51.130:56533 (local address 127.21.51.190:42871)
0809 16:21:03.669919 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0809 16:21:03.669921 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:21:03.669931 (+ 10us) server_negotiation.cc:407] Connection header received
0809 16:21:03.669962 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:21:03.669962 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:21:03.669973 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:21:03.670028 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:21:03.670387 (+ 359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:21:03.670905 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:21:03.671654 (+ 749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:21:03.671860 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:21:03.671919 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:21:03.674524 (+ 2605us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.51.130:56533: BlockingRecv error: recv got EOF from 127.21.51.130:56533 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
Aug 09 16:21:03 dist-test-slave-s11t krb5kdc[1930](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.51.131@KRBTEST.COM for oryx/127.21.51.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 16:21:03 dist-test-slave-s11t krb5kdc[1930](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.51.131@KRBTEST.COM for oryx/127.21.51.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 16:21:03.807689 3648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:21:03.802824 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:42871 (local address 127.21.51.131:41563)
0809 16:21:03.802996 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0809 16:21:03.803008 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 16:21:03.803071 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:21:03.803196 (+ 125us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:21:03.803197 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:21:03.803364 (+ 167us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:21:03.803434 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:21:03.803435 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:21:03.804130 (+ 695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:21:03.804130 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:21:03.804658 (+ 528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:21:03.804660 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:21:03.805092 (+ 432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:21:03.805093 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:21:03.805185 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:21:03.805654 (+ 469us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:21:03.805659 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:21:03.807586 (+ 1927us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.51.190:42871: Server oryx/127.21.51.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":90,"threads_started":1}
W20260809 16:21:03.807832 3645 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:21:03.802911 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.51.131:41563 (local address 127.21.51.190:42871)
0809 16:21:03.803003 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0809 16:21:03.803006 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 16:21:03.803067 (+ 61us) server_negotiation.cc:407] Connection header received
0809 16:21:03.803114 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:21:03.803114 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:21:03.803127 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:21:03.803178 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:21:03.803492 (+ 314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:21:03.804051 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:21:03.804804 (+ 753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:21:03.805022 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:21:03.805076 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:21:03.807760 (+ 2684us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.51.131:41563: BlockingRecv error: recv got EOF from 127.21.51.131:41563 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
Aug 09 16:21:04 dist-test-slave-s11t krb5kdc[1930](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.51.129@KRBTEST.COM for oryx/127.21.51.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 16:21:04 dist-test-slave-s11t krb5kdc[1930](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.51.129@KRBTEST.COM for oryx/127.21.51.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 16:21:04.546789 3649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:21:04.542050 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:42871 (local address 127.21.51.129:36019)
0809 16:21:04.542198 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0809 16:21:04.542210 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 16:21:04.542262 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:21:04.542491 (+ 229us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:21:04.542492 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:21:04.542663 (+ 171us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:21:04.542726 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:21:04.542727 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:21:04.543375 (+ 648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:21:04.543376 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:21:04.544020 (+ 644us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:21:04.544024 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:21:04.544356 (+ 332us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:21:04.544356 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:21:04.544404 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:21:04.544933 (+ 529us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:21:04.544938 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:21:04.546729 (+ 1791us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.51.190:42871: Server oryx/127.21.51.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":85,"threads_started":1}
W20260809 16:21:04.546972 3651 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:21:04.542196 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.51.129:36019 (local address 127.21.51.190:42871)
0809 16:21:04.542358 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0809 16:21:04.542360 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:21:04.542369 (+ 9us) server_negotiation.cc:407] Connection header received
0809 16:21:04.542392 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:21:04.542392 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:21:04.542402 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:21:04.542469 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:21:04.542793 (+ 324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:21:04.543295 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:21:04.544091 (+ 796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:21:04.544293 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:21:04.544345 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:21:04.546898 (+ 2553us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.51.129:36019: BlockingRecv error: recv got EOF from 127.21.51.129:36019 (error 108)
Metrics: {"server-negotiator.queue_time_us":134,"thread_start_us":107,"threads_started":1}
Aug 09 16:21:04 dist-test-slave-s11t krb5kdc[1930](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.51.130@KRBTEST.COM for oryx/127.21.51.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 16:21:04 dist-test-slave-s11t krb5kdc[1930](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.51.130@KRBTEST.COM for oryx/127.21.51.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 16:21:04.679507 3652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:21:04.674917 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:42871 (local address 127.21.51.130:35915)
0809 16:21:04.675052 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0809 16:21:04.675064 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 16:21:04.675133 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:21:04.675275 (+ 142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:21:04.675276 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:21:04.675452 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:21:04.675517 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:21:04.675519 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:21:04.676182 (+ 663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:21:04.676183 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:21:04.676734 (+ 551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:21:04.676736 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:21:04.677085 (+ 349us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:21:04.677086 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:21:04.677144 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:21:04.677596 (+ 452us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:21:04.677600 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:21:04.679448 (+ 1848us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.51.190:42871: Server oryx/127.21.51.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":80,"threads_started":1}
W20260809 16:21:04.679631 3651 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:21:04.675014 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.51.130:35915 (local address 127.21.51.190:42871)
0809 16:21:04.675076 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0809 16:21:04.675078 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:21:04.675137 (+ 59us) server_negotiation.cc:407] Connection header received
0809 16:21:04.675201 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:21:04.675201 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:21:04.675212 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:21:04.675255 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:21:04.675587 (+ 332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:21:04.676106 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:21:04.676820 (+ 714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:21:04.677014 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:21:04.677069 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:21:04.679551 (+ 2482us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.51.130:35915: BlockingRecv error: recv got EOF from 127.21.51.130:35915 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
Aug 09 16:21:04 dist-test-slave-s11t krb5kdc[1930](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.51.131@KRBTEST.COM for oryx/127.21.51.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 16:21:04 dist-test-slave-s11t krb5kdc[1930](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.51.131@KRBTEST.COM for oryx/127.21.51.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 16:21:04.813097 3653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:21:04.808246 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:42871 (local address 127.21.51.131:60159)
0809 16:21:04.808429 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0809 16:21:04.808439 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 16:21:04.808527 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:21:04.808781 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:21:04.808782 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:21:04.808946 (+ 164us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:21:04.808996 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:21:04.808998 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:21:04.809668 (+ 670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:21:04.809670 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:21:04.810236 (+ 566us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:21:04.810238 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:21:04.810619 (+ 381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:21:04.810620 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:21:04.810688 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:21:04.811176 (+ 488us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:21:04.811180 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:21:04.812998 (+ 1818us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.51.190:42871: Server oryx/127.21.51.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":98,"threads_started":1}
W20260809 16:21:04.813135 3651 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:21:04.808341 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.51.131:60159 (local address 127.21.51.190:42871)
0809 16:21:04.808390 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0809 16:21:04.808392 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:21:04.808505 (+ 113us) server_negotiation.cc:407] Connection header received
0809 16:21:04.808624 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:21:04.808625 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:21:04.808641 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:21:04.808716 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:21:04.809080 (+ 364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:21:04.809588 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:21:04.810327 (+ 739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:21:04.810547 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:21:04.810622 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:21:04.813075 (+ 2453us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.51.131:60159: BlockingRecv error: recv got EOF from 127.21.51.131:60159 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
Aug 09 16:21:05 dist-test-slave-s11t krb5kdc[1930](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.51.129@KRBTEST.COM for oryx/127.21.51.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 16:21:05 dist-test-slave-s11t krb5kdc[1930](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.51.129@KRBTEST.COM for oryx/127.21.51.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 16:21:05.552556 3654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:21:05.547297 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:42871 (local address 127.21.51.129:52057)
0809 16:21:05.547450 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0809 16:21:05.547462 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 16:21:05.547521 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:21:05.547691 (+ 170us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:21:05.547693 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:21:05.547884 (+ 191us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:21:05.547937 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:21:05.547939 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:21:05.548651 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:21:05.548652 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:21:05.549320 (+ 668us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:21:05.549323 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:21:05.549847 (+ 524us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:21:05.549848 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:21:05.549910 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:21:05.550370 (+ 460us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:21:05.550375 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:21:05.552457 (+ 2082us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.51.190:42871: Server oryx/127.21.51.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":94,"threads_started":1}
W20260809 16:21:05.552618 3655 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:21:05.547393 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.51.129:52057 (local address 127.21.51.190:42871)
0809 16:21:05.547520 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0809 16:21:05.547522 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:21:05.547531 (+ 9us) server_negotiation.cc:407] Connection header received
0809 16:21:05.547574 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:21:05.547575 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:21:05.547586 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:21:05.547638 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:21:05.548011 (+ 373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:21:05.548560 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:21:05.549430 (+ 870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:21:05.549759 (+ 329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:21:05.549845 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:21:05.552541 (+ 2696us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.51.129:52057: BlockingRecv error: recv got EOF from 127.21.51.129:52057 (error 108)
Metrics: {"server-negotiator.queue_time_us":102,"thread_start_us":74,"threads_started":1}
Aug 09 16:21:05 dist-test-slave-s11t krb5kdc[1930](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.51.130@KRBTEST.COM for oryx/127.21.51.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 16:21:05 dist-test-slave-s11t krb5kdc[1930](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.51.130@KRBTEST.COM for oryx/127.21.51.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 16:21:05.684952 3657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:21:05.680115 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:42871 (local address 127.21.51.130:38087)
0809 16:21:05.680303 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0809 16:21:05.680317 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 16:21:05.680415 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:21:05.680571 (+ 156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:21:05.680572 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:21:05.680784 (+ 212us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:21:05.680835 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:21:05.680838 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:21:05.681585 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:21:05.681587 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:21:05.682120 (+ 533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:21:05.682122 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:21:05.682512 (+ 390us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:21:05.682513 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:21:05.682579 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:21:05.683072 (+ 493us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:21:05.683076 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:21:05.684891 (+ 1815us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.51.190:42871: Server oryx/127.21.51.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":90,"threads_started":1}
W20260809 16:21:05.685084 3655 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:21:05.680217 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.51.130:38087 (local address 127.21.51.190:42871)
0809 16:21:05.680306 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0809 16:21:05.680309 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 16:21:05.680394 (+ 85us) server_negotiation.cc:407] Connection header received
0809 16:21:05.680507 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:21:05.680508 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:21:05.680519 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:21:05.680568 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:21:05.680962 (+ 394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:21:05.681489 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:21:05.682207 (+ 718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:21:05.682425 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:21:05.682503 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:21:05.685002 (+ 2499us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.51.130:38087: BlockingRecv error: recv got EOF from 127.21.51.130:38087 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Aug 09 16:21:05 dist-test-slave-s11t krb5kdc[1930](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.51.131@KRBTEST.COM for oryx/127.21.51.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 16:21:05 dist-test-slave-s11t krb5kdc[1930](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.51.131@KRBTEST.COM for oryx/127.21.51.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 16:21:05.818238 3660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:21:05.813577 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:42871 (local address 127.21.51.131:39979)
0809 16:21:05.813739 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0809 16:21:05.813749 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 16:21:05.813805 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:21:05.813930 (+ 125us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:21:05.813931 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:21:05.814088 (+ 157us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:21:05.814139 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:21:05.814141 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:21:05.814777 (+ 636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:21:05.814778 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:21:05.815339 (+ 561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:21:05.815341 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:21:05.815659 (+ 318us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:21:05.815660 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:21:05.815704 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:21:05.816235 (+ 531us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:21:05.816240 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:21:05.818149 (+ 1909us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.51.190:42871: Server oryx/127.21.51.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":97,"threads_started":1}
W20260809 16:21:05.818377 3655 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:21:05.813658 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.51.131:39979 (local address 127.21.51.190:42871)
0809 16:21:05.813706 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0809 16:21:05.813708 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:21:05.813793 (+ 85us) server_negotiation.cc:407] Connection header received
0809 16:21:05.813860 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:21:05.813861 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:21:05.813875 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:21:05.813926 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:21:05.814198 (+ 272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:21:05.814704 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:21:05.815415 (+ 711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:21:05.815593 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:21:05.815642 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:21:05.818303 (+ 2661us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.51.131:39979: BlockingRecv error: recv got EOF from 127.21.51.131:39979 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
Aug 09 16:21:06 dist-test-slave-s11t krb5kdc[1930](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.51.129@KRBTEST.COM for oryx/127.21.51.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 16:21:06 dist-test-slave-s11t krb5kdc[1930](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.51.129@KRBTEST.COM for oryx/127.21.51.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 16:21:06.557809 3661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:21:06.553058 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:42871 (local address 127.21.51.129:56977)
0809 16:21:06.553202 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0809 16:21:06.553215 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 16:21:06.553278 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:21:06.553405 (+ 127us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:21:06.553405 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:21:06.553571 (+ 166us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:21:06.553623 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:21:06.553624 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:21:06.554337 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:21:06.554337 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:21:06.554876 (+ 539us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:21:06.554878 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:21:06.555303 (+ 425us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:21:06.555303 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:21:06.555344 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:21:06.555842 (+ 498us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:21:06.555846 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:21:06.557749 (+ 1903us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.51.190:42871: Server oryx/127.21.51.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":67,"threads_started":1}
W20260809 16:21:06.557962 3662 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:21:06.553160 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.51.129:56977 (local address 127.21.51.190:42871)
0809 16:21:06.553292 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0809 16:21:06.553293 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 16:21:06.553301 (+ 8us) server_negotiation.cc:407] Connection header received
0809 16:21:06.553324 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:21:06.553324 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:21:06.553335 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:21:06.553393 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:21:06.553727 (+ 334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:21:06.554256 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:21:06.554983 (+ 727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:21:06.555223 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:21:06.555298 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:21:06.557872 (+ 2574us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.51.129:56977: BlockingRecv error: recv got EOF from 127.21.51.129:56977 (error 108)
Metrics: {"server-negotiator.queue_time_us":112,"thread_start_us":68,"threads_started":1}
Aug 09 16:21:06 dist-test-slave-s11t krb5kdc[1930](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.51.130@KRBTEST.COM for oryx/127.21.51.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 16:21:06 dist-test-slave-s11t krb5kdc[1930](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.51.130@KRBTEST.COM for oryx/127.21.51.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 16:21:06.690294 3663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:21:06.685511 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:42871 (local address 127.21.51.130:39801)
0809 16:21:06.685776 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0809 16:21:06.685790 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 16:21:06.685860 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:21:06.686001 (+ 141us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:21:06.686001 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:21:06.686175 (+ 174us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:21:06.686237 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:21:06.686239 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:21:06.687038 (+ 799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:21:06.687039 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:21:06.687585 (+ 546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:21:06.687587 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:21:06.687920 (+ 333us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:21:06.687921 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:21:06.687968 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:21:06.688422 (+ 454us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:21:06.688426 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:21:06.690237 (+ 1811us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.51.190:42871: Server oryx/127.21.51.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":245,"thread_start_us":216,"threads_started":1}
W20260809 16:21:06.690410 3662 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:21:06.685588 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.51.130:39801 (local address 127.21.51.190:42871)
0809 16:21:06.685659 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0809 16:21:06.685660 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 16:21:06.685856 (+ 196us) server_negotiation.cc:407] Connection header received
0809 16:21:06.685912 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:21:06.685913 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:21:06.685925 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:21:06.685978 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:21:06.686300 (+ 322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:21:06.686962 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:21:06.687662 (+ 700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:21:06.687847 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:21:06.687900 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:21:06.690335 (+ 2435us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.51.130:39801: BlockingRecv error: recv got EOF from 127.21.51.130:39801 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
Aug 09 16:21:06 dist-test-slave-s11t krb5kdc[1930](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.51.131@KRBTEST.COM for oryx/127.21.51.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 16:21:06 dist-test-slave-s11t krb5kdc[1930](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.51.131@KRBTEST.COM for oryx/127.21.51.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 16:21:06.824041 3666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:21:06.818787 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:42871 (local address 127.21.51.131:41529)
0809 16:21:06.818973 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0809 16:21:06.818985 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 16:21:06.819049 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:21:06.819196 (+ 147us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:21:06.819197 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:21:06.819366 (+ 169us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:21:06.819416 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:21:06.819417 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:21:06.820228 (+ 811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:21:06.820229 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:21:06.821118 (+ 889us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:21:06.821121 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:21:06.821449 (+ 328us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:21:06.821450 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:21:06.821495 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:21:06.822178 (+ 683us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:21:06.822186 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:21:06.823982 (+ 1796us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.51.190:42871: Server oryx/127.21.51.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":109,"threads_started":1}
W20260809 16:21:06.824163 3662 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:21:06.818834 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.51.131:41529 (local address 127.21.51.190:42871)
0809 16:21:06.818923 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0809 16:21:06.818925 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:21:06.819047 (+ 122us) server_negotiation.cc:407] Connection header received
0809 16:21:06.819110 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:21:06.819110 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:21:06.819124 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:21:06.819176 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:21:06.819475 (+ 299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:21:06.820113 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:21:06.821191 (+ 1078us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:21:06.821374 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:21:06.821433 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:21:06.824083 (+ 2650us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.51.131:41529: BlockingRecv error: recv got EOF from 127.21.51.131:41529 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
Aug 09 16:21:07 dist-test-slave-s11t krb5kdc[1930](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.51.129@KRBTEST.COM for oryx/127.21.51.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 16:21:07 dist-test-slave-s11t krb5kdc[1930](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.51.129@KRBTEST.COM for oryx/127.21.51.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 16:21:07.568172 3668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:21:07.558314 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:42871 (local address 127.21.51.129:38875)
0809 16:21:07.558505 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0809 16:21:07.558520 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 16:21:07.558622 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:21:07.558808 (+ 186us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:21:07.558809 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:21:07.559061 (+ 252us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:21:07.559129 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:21:07.559132 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:21:07.559823 (+ 691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:21:07.559824 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:21:07.560410 (+ 586us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:21:07.560412 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:21:07.560823 (+ 411us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:21:07.560825 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:21:07.560876 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:21:07.566112 (+ 5236us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:21:07.566118 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:21:07.568110 (+ 1992us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.51.190:42871: Server oryx/127.21.51.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":92,"threads_started":1}
W20260809 16:21:07.568397 3669 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:21:07.558414 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.51.129:38875 (local address 127.21.51.190:42871)
0809 16:21:07.558637 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0809 16:21:07.558639 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:21:07.558648 (+ 9us) server_negotiation.cc:407] Connection header received
0809 16:21:07.558677 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:21:07.558677 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:21:07.558707 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:21:07.558783 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:21:07.559225 (+ 442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:21:07.559746 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:21:07.560490 (+ 744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:21:07.560743 (+ 253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:21:07.560801 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:21:07.568315 (+ 7514us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.51.129:38875: BlockingRecv error: recv got EOF from 127.21.51.129:38875 (error 108)
Metrics: {"server-negotiator.queue_time_us":200,"thread_start_us":92,"threads_started":1}
Aug 09 16:21:07 dist-test-slave-s11t krb5kdc[1930](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.51.130@KRBTEST.COM for oryx/127.21.51.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 16:21:07 dist-test-slave-s11t krb5kdc[1930](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.51.130@KRBTEST.COM for oryx/127.21.51.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 16:21:07.695675 3672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:21:07.690826 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:42871 (local address 127.21.51.130:33575)
0809 16:21:07.691036 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0809 16:21:07.691048 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 16:21:07.691118 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:21:07.691279 (+ 161us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:21:07.691280 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:21:07.691466 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:21:07.691518 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:21:07.691519 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:21:07.692143 (+ 624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:21:07.692144 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:21:07.692782 (+ 638us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:21:07.692785 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:21:07.693127 (+ 342us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:21:07.693128 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:21:07.693182 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:21:07.693718 (+ 536us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:21:07.693724 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:21:07.695616 (+ 1892us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.51.190:42871: Server oryx/127.21.51.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":137,"threads_started":1}
W20260809 16:21:07.695816 3669 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:21:07.690931 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.51.130:33575 (local address 127.21.51.190:42871)
0809 16:21:07.691020 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0809 16:21:07.691023 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 16:21:07.691103 (+ 80us) server_negotiation.cc:407] Connection header received
0809 16:21:07.691188 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:21:07.691188 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:21:07.691200 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:21:07.691254 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:21:07.691585 (+ 331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:21:07.692070 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:21:07.692852 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:21:07.693035 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:21:07.693087 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:21:07.695732 (+ 2645us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.51.130:33575: BlockingRecv error: recv got EOF from 127.21.51.130:33575 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
Aug 09 16:21:07 dist-test-slave-s11t krb5kdc[1930](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.51.131@KRBTEST.COM for oryx/127.21.51.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 16:21:07 dist-test-slave-s11t krb5kdc[1930](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.51.131@KRBTEST.COM for oryx/127.21.51.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 16:21:07.829383 3674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:21:07.824545 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:42871 (local address 127.21.51.131:48699)
0809 16:21:07.824693 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0809 16:21:07.824725 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0809 16:21:07.824813 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:21:07.824977 (+ 164us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:21:07.824978 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:21:07.825146 (+ 168us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:21:07.825198 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:21:07.825199 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:21:07.825826 (+ 627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:21:07.825827 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:21:07.826353 (+ 526us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:21:07.826354 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:21:07.826693 (+ 339us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:21:07.826694 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:21:07.826757 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:21:07.827215 (+ 458us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:21:07.827219 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:21:07.829319 (+ 2100us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.51.190:42871: Server oryx/127.21.51.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":70,"threads_started":1}
W20260809 16:21:07.829519 3669 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:21:07.824657 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.51.131:48699 (local address 127.21.51.190:42871)
0809 16:21:07.824751 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0809 16:21:07.824753 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:21:07.824796 (+ 43us) server_negotiation.cc:407] Connection header received
0809 16:21:07.824870 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:21:07.824871 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:21:07.824885 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:21:07.824952 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:21:07.825263 (+ 311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:21:07.825753 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:21:07.826428 (+ 675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:21:07.826607 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:21:07.826657 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:21:07.829434 (+ 2777us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.51.131:48699: BlockingRecv error: recv got EOF from 127.21.51.131:48699 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
Aug 09 16:21:08 dist-test-slave-s11t krb5kdc[1930](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.51.129@KRBTEST.COM for oryx/127.21.51.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 16:21:08 dist-test-slave-s11t krb5kdc[1930](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.51.129@KRBTEST.COM for oryx/127.21.51.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 16:21:08.573436 3676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:21:08.568672 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:42871 (local address 127.21.51.129:51777)
0809 16:21:08.568919 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0809 16:21:08.568930 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 16:21:08.568989 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:21:08.569130 (+ 141us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:21:08.569131 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:21:08.569316 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:21:08.569368 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:21:08.569370 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:21:08.570025 (+ 655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:21:08.570025 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:21:08.570568 (+ 543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:21:08.570570 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:21:08.570893 (+ 323us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:21:08.570894 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:21:08.570939 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:21:08.571394 (+ 455us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:21:08.571398 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:21:08.573341 (+ 1943us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.51.190:42871: Server oryx/127.21.51.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":223,"thread_start_us":90,"threads_started":1}
W20260809 16:21:08.573518 3678 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:21:08.568779 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.51.129:51777 (local address 127.21.51.190:42871)
0809 16:21:08.568940 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0809 16:21:08.568942 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:21:08.568974 (+ 32us) server_negotiation.cc:407] Connection header received
0809 16:21:08.569045 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:21:08.569045 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:21:08.569055 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:21:08.569105 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:21:08.569436 (+ 331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:21:08.569956 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:21:08.570640 (+ 684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:21:08.570827 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:21:08.570889 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:21:08.573437 (+ 2548us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.51.129:51777: BlockingRecv error: recv got EOF from 127.21.51.129:51777 (error 108)
Metrics: {"server-negotiator.queue_time_us":133,"thread_start_us":95,"threads_started":1}
Aug 09 16:21:08 dist-test-slave-s11t krb5kdc[1930](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.51.130@KRBTEST.COM for oryx/127.21.51.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 16:21:08 dist-test-slave-s11t krb5kdc[1930](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.51.130@KRBTEST.COM for oryx/127.21.51.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 16:21:08.701138 3680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:21:08.696222 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:42871 (local address 127.21.51.130:35359)
0809 16:21:08.696378 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0809 16:21:08.696390 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 16:21:08.696459 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:21:08.696588 (+ 129us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:21:08.696589 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:21:08.696800 (+ 211us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:21:08.696856 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:21:08.696857 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:21:08.697501 (+ 644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:21:08.697502 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:21:08.698113 (+ 611us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:21:08.698114 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:21:08.698493 (+ 379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:21:08.698494 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:21:08.698606 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:21:08.699109 (+ 503us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:21:08.699114 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:21:08.701075 (+ 1961us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.51.190:42871: Server oryx/127.21.51.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":85,"threads_started":1}
W20260809 16:21:08.701305 3678 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:21:08.696330 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.51.130:35359 (local address 127.21.51.190:42871)
0809 16:21:08.696391 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0809 16:21:08.696393 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:21:08.696440 (+ 47us) server_negotiation.cc:407] Connection header received
0809 16:21:08.696513 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:21:08.696514 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:21:08.696525 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:21:08.696567 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:21:08.696926 (+ 359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:21:08.697425 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:21:08.698218 (+ 793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:21:08.698415 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:21:08.698481 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:21:08.701222 (+ 2741us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.51.130:35359: BlockingRecv error: recv got EOF from 127.21.51.130:35359 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
Aug 09 16:21:08 dist-test-slave-s11t krb5kdc[1930](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.51.131@KRBTEST.COM for oryx/127.21.51.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 16:21:08 dist-test-slave-s11t krb5kdc[1930](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.51.131@KRBTEST.COM for oryx/127.21.51.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 16:21:08.835225 3682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:21:08.829818 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:42871 (local address 127.21.51.131:40689)
0809 16:21:08.829967 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0809 16:21:08.829981 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 16:21:08.830057 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:21:08.830224 (+ 167us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:21:08.830225 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:21:08.830415 (+ 190us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:21:08.830464 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:21:08.830466 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:21:08.831203 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:21:08.831204 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:21:08.832147 (+ 943us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:21:08.832151 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:21:08.832676 (+ 525us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:21:08.832677 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:21:08.832761 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:21:08.833261 (+ 500us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:21:08.833266 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:21:08.835117 (+ 1851us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.51.190:42871: Server oryx/127.21.51.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":79,"threads_started":1}
W20260809 16:21:08.835265 3678 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:21:08.829953 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.51.131:40689 (local address 127.21.51.190:42871)
0809 16:21:08.829999 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0809 16:21:08.830003 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 16:21:08.830036 (+ 33us) server_negotiation.cc:407] Connection header received
0809 16:21:08.830120 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:21:08.830121 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:21:08.830138 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:21:08.830204 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:21:08.830544 (+ 340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:21:08.831124 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:21:08.832242 (+ 1118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:21:08.832614 (+ 372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:21:08.832677 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:21:08.835197 (+ 2520us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.51.131:40689: BlockingRecv error: recv got EOF from 127.21.51.131:40689 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
Aug 09 16:21:09 dist-test-slave-s11t krb5kdc[1930](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.51.129@KRBTEST.COM for oryx/127.21.51.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 16:21:09 dist-test-slave-s11t krb5kdc[1930](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.51.129@KRBTEST.COM for oryx/127.21.51.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 16:21:09.578905 3683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:21:09.574078 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:42871 (local address 127.21.51.129:36233)
0809 16:21:09.574261 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0809 16:21:09.574272 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 16:21:09.574332 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:21:09.574488 (+ 156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:21:09.574489 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:21:09.574666 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:21:09.574717 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:21:09.574719 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:21:09.575458 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:21:09.575460 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:21:09.576175 (+ 715us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:21:09.576178 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:21:09.576523 (+ 345us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:21:09.576523 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:21:09.576565 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:21:09.577038 (+ 473us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:21:09.577042 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:21:09.578833 (+ 1791us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.51.190:42871: Server oryx/127.21.51.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":102,"threads_started":1}
W20260809 16:21:09.579006 3684 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:21:09.574182 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.51.129:36233 (local address 127.21.51.190:42871)
0809 16:21:09.574299 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0809 16:21:09.574301 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:21:09.574314 (+ 13us) server_negotiation.cc:407] Connection header received
0809 16:21:09.574398 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:21:09.574398 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:21:09.574408 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:21:09.574470 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:21:09.574822 (+ 352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:21:09.575379 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:21:09.576260 (+ 881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:21:09.576464 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:21:09.576516 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:21:09.578929 (+ 2413us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.51.129:36233: BlockingRecv error: recv got EOF from 127.21.51.129:36233 (error 108)
Metrics: {"server-negotiator.queue_time_us":84,"thread_start_us":56,"threads_started":1}
Aug 09 16:21:09 dist-test-slave-s11t krb5kdc[1930](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.51.130@KRBTEST.COM for oryx/127.21.51.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 16:21:09 dist-test-slave-s11t krb5kdc[1930](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.51.130@KRBTEST.COM for oryx/127.21.51.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 16:21:09.706528 3688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:21:09.701668 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:42871 (local address 127.21.51.130:54751)
0809 16:21:09.701820 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0809 16:21:09.701836 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 16:21:09.701895 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:21:09.702024 (+ 129us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:21:09.702025 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:21:09.702202 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:21:09.702263 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:21:09.702264 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:21:09.702908 (+ 644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:21:09.702910 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:21:09.703446 (+ 536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:21:09.703448 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:21:09.703816 (+ 368us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:21:09.703818 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:21:09.703919 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:21:09.704449 (+ 530us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:21:09.704454 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:21:09.706453 (+ 1999us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.51.190:42871: Server oryx/127.21.51.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":79,"threads_started":1}
W20260809 16:21:09.706643 3684 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:21:09.701749 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.51.130:54751 (local address 127.21.51.190:42871)
0809 16:21:09.701837 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0809 16:21:09.701839 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:21:09.701933 (+ 94us) server_negotiation.cc:407] Connection header received
0809 16:21:09.701963 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:21:09.701963 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:21:09.701976 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:21:09.702020 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:21:09.702319 (+ 299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:21:09.702830 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:21:09.703515 (+ 685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:21:09.703719 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:21:09.703770 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:21:09.706551 (+ 2781us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.51.130:54751: BlockingRecv error: recv got EOF from 127.21.51.130:54751 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
Aug 09 16:21:09 dist-test-slave-s11t krb5kdc[1930](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.51.131@KRBTEST.COM for oryx/127.21.51.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 16:21:09 dist-test-slave-s11t krb5kdc[1930](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.51.131@KRBTEST.COM for oryx/127.21.51.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 16:21:09.840514 3689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:21:09.835765 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:42871 (local address 127.21.51.131:38533)
0809 16:21:09.835915 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0809 16:21:09.835927 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 16:21:09.836011 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:21:09.836147 (+ 136us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:21:09.836148 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:21:09.836344 (+ 196us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:21:09.836394 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:21:09.836396 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:21:09.837131 (+ 735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:21:09.837132 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:21:09.837670 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:21:09.837671 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:21:09.838026 (+ 355us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:21:09.838027 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:21:09.838084 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:21:09.838542 (+ 458us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:21:09.838547 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:21:09.840443 (+ 1896us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.51.190:42871: Server oryx/127.21.51.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":90,"threads_started":1}
W20260809 16:21:09.840631 3684 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:21:09.835824 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.51.131:38533 (local address 127.21.51.190:42871)
0809 16:21:09.835891 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0809 16:21:09.835893 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:21:09.835996 (+ 103us) server_negotiation.cc:407] Connection header received
0809 16:21:09.836068 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:21:09.836069 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:21:09.836085 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:21:09.836132 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:21:09.836473 (+ 341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:21:09.837048 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:21:09.837757 (+ 709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:21:09.837952 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:21:09.838028 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:21:09.840538 (+ 2510us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.51.131:38533: BlockingRecv error: recv got EOF from 127.21.51.131:38533 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
Aug 09 16:21:10 dist-test-slave-s11t krb5kdc[1930](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.51.129@KRBTEST.COM for oryx/127.21.51.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 16:21:10 dist-test-slave-s11t krb5kdc[1930](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.51.129@KRBTEST.COM for oryx/127.21.51.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 16:21:10.584631 3690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:21:10.579502 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:42871 (local address 127.21.51.129:57315)
0809 16:21:10.579692 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0809 16:21:10.579705 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 16:21:10.579788 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:21:10.579932 (+ 144us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:21:10.579933 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:21:10.580111 (+ 178us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:21:10.580170 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:21:10.580172 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:21:10.580960 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:21:10.580961 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:21:10.581514 (+ 553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:21:10.581516 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:21:10.581900 (+ 384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:21:10.581901 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:21:10.581950 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:21:10.582479 (+ 529us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:21:10.582485 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:21:10.584569 (+ 2084us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.51.190:42871: Server oryx/127.21.51.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":89,"threads_started":1}
W20260809 16:21:10.584824 3691 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:21:10.579627 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.51.129:57315 (local address 127.21.51.190:42871)
0809 16:21:10.579776 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0809 16:21:10.579778 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:21:10.579788 (+ 10us) server_negotiation.cc:407] Connection header received
0809 16:21:10.579837 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:21:10.579838 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:21:10.579849 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:21:10.579913 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:21:10.580275 (+ 362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:21:10.580872 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:21:10.581592 (+ 720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:21:10.581825 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:21:10.581886 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:21:10.584718 (+ 2832us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.51.129:57315: BlockingRecv error: recv got EOF from 127.21.51.129:57315 (error 108)
Metrics: {"server-negotiator.queue_time_us":127,"thread_start_us":76,"threads_started":1}
Aug 09 16:21:10 dist-test-slave-s11t krb5kdc[1930](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.51.130@KRBTEST.COM for oryx/127.21.51.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 16:21:10 dist-test-slave-s11t krb5kdc[1930](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.51.130@KRBTEST.COM for oryx/127.21.51.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 16:21:10.711980 3692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:21:10.706952 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:42871 (local address 127.21.51.130:50037)
0809 16:21:10.707123 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0809 16:21:10.707156 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0809 16:21:10.707236 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:21:10.707380 (+ 144us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:21:10.707380 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:21:10.707548 (+ 168us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:21:10.707601 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:21:10.707602 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:21:10.708380 (+ 778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:21:10.708381 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:21:10.709004 (+ 623us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:21:10.709007 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:21:10.709354 (+ 347us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:21:10.709355 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:21:10.709402 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:21:10.709929 (+ 527us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:21:10.709934 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:21:10.711891 (+ 1957us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.51.190:42871: Server oryx/127.21.51.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":88,"threads_started":1}
W20260809 16:21:10.712100 3691 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:21:10.707044 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.51.130:50037 (local address 127.21.51.190:42871)
0809 16:21:10.707121 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0809 16:21:10.707123 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:21:10.707213 (+ 90us) server_negotiation.cc:407] Connection header received
0809 16:21:10.707309 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:21:10.707310 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:21:10.707322 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:21:10.707372 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:21:10.707689 (+ 317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:21:10.708305 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:21:10.709081 (+ 776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:21:10.709287 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:21:10.709345 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:21:10.712033 (+ 2688us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.51.130:50037: BlockingRecv error: recv got EOF from 127.21.51.130:50037 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
Aug 09 16:21:10 dist-test-slave-s11t krb5kdc[1930](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.51.131@KRBTEST.COM for oryx/127.21.51.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 16:21:10 dist-test-slave-s11t krb5kdc[1930](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.51.131@KRBTEST.COM for oryx/127.21.51.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 16:21:10.845723 3696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:21:10.841036 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:42871 (local address 127.21.51.131:43161)
0809 16:21:10.841174 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0809 16:21:10.841191 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 16:21:10.841256 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:21:10.841397 (+ 141us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:21:10.841398 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:21:10.841588 (+ 190us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:21:10.841635 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:21:10.841636 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:21:10.842349 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:21:10.842350 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:21:10.842891 (+ 541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:21:10.842893 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:21:10.843271 (+ 378us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:21:10.843272 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:21:10.843345 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:21:10.843779 (+ 434us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:21:10.843784 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:21:10.845665 (+ 1881us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.51.190:42871: Server oryx/127.21.51.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":85,"threads_started":1}
W20260809 16:21:10.845919 3691 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:21:10.841155 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.51.131:43161 (local address 127.21.51.190:42871)
0809 16:21:10.841238 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0809 16:21:10.841241 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 16:21:10.841251 (+ 10us) server_negotiation.cc:407] Connection header received
0809 16:21:10.841315 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:21:10.841316 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:21:10.841333 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:21:10.841379 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:21:10.841717 (+ 338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:21:10.842239 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:21:10.842982 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:21:10.843195 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:21:10.843273 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:21:10.845860 (+ 2587us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.51.131:43161: BlockingRecv error: recv got EOF from 127.21.51.131:43161 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
Aug 09 16:21:11 dist-test-slave-s11t krb5kdc[1930](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.51.129@KRBTEST.COM for oryx/127.21.51.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 16:21:11 dist-test-slave-s11t krb5kdc[1930](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.51.129@KRBTEST.COM for oryx/127.21.51.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 16:21:11.590137 3698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:21:11.585179 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:42871 (local address 127.21.51.129:54255)
0809 16:21:11.585324 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0809 16:21:11.585338 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 16:21:11.585413 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:21:11.585536 (+ 123us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:21:11.585537 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:21:11.585715 (+ 178us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:21:11.585767 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:21:11.585768 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:21:11.586431 (+ 663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:21:11.586432 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:21:11.587304 (+ 872us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:21:11.587307 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:21:11.587653 (+ 346us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:21:11.587653 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:21:11.587694 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:21:11.588210 (+ 516us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:21:11.588216 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:21:11.590067 (+ 1851us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.51.190:42871: Server oryx/127.21.51.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":80,"threads_started":1}
W20260809 16:21:11.590312 3699 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:21:11.585260 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.51.129:54255 (local address 127.21.51.190:42871)
0809 16:21:11.585403 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0809 16:21:11.585405 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:21:11.585415 (+ 10us) server_negotiation.cc:407] Connection header received
0809 16:21:11.585470 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:21:11.585471 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:21:11.585482 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:21:11.585531 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:21:11.585833 (+ 302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:21:11.586346 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:21:11.587380 (+ 1034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:21:11.587588 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:21:11.587644 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:21:11.590206 (+ 2562us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.51.129:54255: BlockingRecv error: recv got EOF from 127.21.51.129:54255 (error 108)
Metrics: {"server-negotiator.queue_time_us":122,"thread_start_us":75,"threads_started":1}
Aug 09 16:21:11 dist-test-slave-s11t krb5kdc[1930](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.51.130@KRBTEST.COM for oryx/127.21.51.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 16:21:11 dist-test-slave-s11t krb5kdc[1930](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.51.130@KRBTEST.COM for oryx/127.21.51.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 16:21:11.717712 3700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:21:11.712672 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:42871 (local address 127.21.51.130:57741)
0809 16:21:11.712831 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0809 16:21:11.712917 (+ 86us) client_negotiation.cc:174] Beginning negotiation
0809 16:21:11.713006 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:21:11.713205 (+ 199us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:21:11.713206 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:21:11.713463 (+ 257us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:21:11.713547 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:21:11.713551 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:21:11.714213 (+ 662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:21:11.714214 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:21:11.714788 (+ 574us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:21:11.714790 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:21:11.715121 (+ 331us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:21:11.715122 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:21:11.715165 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:21:11.715586 (+ 421us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:21:11.715591 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:21:11.717633 (+ 2042us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.51.190:42871: Server oryx/127.21.51.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":70,"threads_started":1}
W20260809 16:21:11.717839 3699 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:21:11.712819 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.51.130:57741 (local address 127.21.51.190:42871)
0809 16:21:11.712892 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0809 16:21:11.712895 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 16:21:11.713010 (+ 115us) server_negotiation.cc:407] Connection header received
0809 16:21:11.713087 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:21:11.713089 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:21:11.713106 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:21:11.713176 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:21:11.713633 (+ 457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:21:11.714137 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:21:11.714863 (+ 726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:21:11.715055 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:21:11.715110 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:21:11.717737 (+ 2627us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.51.130:57741: BlockingRecv error: recv got EOF from 127.21.51.130:57741 (error 108)
Metrics: {"server-negotiator.queue_time_us":25}
Aug 09 16:21:11 dist-test-slave-s11t krb5kdc[1930](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.51.131@KRBTEST.COM for oryx/127.21.51.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 16:21:11 dist-test-slave-s11t krb5kdc[1930](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.51.131@KRBTEST.COM for oryx/127.21.51.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 16:21:11.851042 3702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:21:11.846354 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:42871 (local address 127.21.51.131:39327)
0809 16:21:11.846497 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0809 16:21:11.846508 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 16:21:11.846566 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:21:11.846683 (+ 117us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:21:11.846683 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:21:11.846875 (+ 192us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:21:11.846925 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:21:11.846927 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:21:11.847589 (+ 662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:21:11.847590 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:21:11.848162 (+ 572us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:21:11.848164 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:21:11.848568 (+ 404us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:21:11.848570 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:21:11.848644 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:21:11.849190 (+ 546us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:21:11.849195 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:21:11.850975 (+ 1780us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.51.190:42871: Server oryx/127.21.51.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":70,"threads_started":1}
W20260809 16:21:11.851236 3699 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:21:11.846406 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.51.131:39327 (local address 127.21.51.190:42871)
0809 16:21:11.846470 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0809 16:21:11.846472 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:21:11.846551 (+ 79us) server_negotiation.cc:407] Connection header received
0809 16:21:11.846619 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:21:11.846619 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:21:11.846634 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:21:11.846670 (+ 36us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:21:11.846989 (+ 319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:21:11.847494 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:21:11.848254 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:21:11.848445 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:21:11.848505 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:21:11.851130 (+ 2625us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.51.131:39327: BlockingRecv error: recv got EOF from 127.21.51.131:39327 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
Aug 09 16:21:12 dist-test-slave-s11t krb5kdc[1930](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.51.129@KRBTEST.COM for oryx/127.21.51.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 16:21:12 dist-test-slave-s11t krb5kdc[1930](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.51.129@KRBTEST.COM for oryx/127.21.51.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 16:21:12.599817 3706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:21:12.590684 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:42871 (local address 127.21.51.129:57147)
0809 16:21:12.590873 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0809 16:21:12.590885 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 16:21:12.590944 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:21:12.591089 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:21:12.591090 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:21:12.591263 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:21:12.591335 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:21:12.591337 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:21:12.592049 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:21:12.592051 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:21:12.592753 (+ 702us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:21:12.592756 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:21:12.593117 (+ 361us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:21:12.593118 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:21:12.593163 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:21:12.597916 (+ 4753us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:21:12.597921 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:21:12.599755 (+ 1834us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.51.190:42871: Server oryx/127.21.51.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":89,"threads_started":1}
W20260809 16:21:12.599993 3707 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:21:12.590787 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.51.129:57147 (local address 127.21.51.190:42871)
0809 16:21:12.590917 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0809 16:21:12.590919 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:21:12.590929 (+ 10us) server_negotiation.cc:407] Connection header received
0809 16:21:12.591007 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:21:12.591007 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:21:12.591018 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:21:12.591068 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:21:12.591405 (+ 337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:21:12.591937 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:21:12.592817 (+ 880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:21:12.593048 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:21:12.593101 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:21:12.599910 (+ 6809us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.51.129:57147: BlockingRecv error: recv got EOF from 127.21.51.129:57147 (error 108)
Metrics: {"server-negotiator.queue_time_us":110,"thread_start_us":57,"threads_started":1}
Aug 09 16:21:12 dist-test-slave-s11t krb5kdc[1930](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.51.130@KRBTEST.COM for oryx/127.21.51.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 16:21:12 dist-test-slave-s11t krb5kdc[1930](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.51.130@KRBTEST.COM for oryx/127.21.51.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 16:21:12.723043 3708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:21:12.718157 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:42871 (local address 127.21.51.130:53951)
0809 16:21:12.718360 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0809 16:21:12.718372 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 16:21:12.718470 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:21:12.718638 (+ 168us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:21:12.718640 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:21:12.718839 (+ 199us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:21:12.718887 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:21:12.718889 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:21:12.719580 (+ 691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:21:12.719581 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:21:12.720172 (+ 591us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:21:12.720174 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:21:12.720550 (+ 376us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:21:12.720551 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:21:12.720617 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:21:12.721096 (+ 479us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:21:12.721100 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:21:12.722984 (+ 1884us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.51.190:42871: Server oryx/127.21.51.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":133,"threads_started":1}
W20260809 16:21:12.723199 3707 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:21:12.718266 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.51.130:53951 (local address 127.21.51.190:42871)
0809 16:21:12.718345 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0809 16:21:12.718347 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:21:12.718447 (+ 100us) server_negotiation.cc:407] Connection header received
0809 16:21:12.718545 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:21:12.718546 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:21:12.718559 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:21:12.718631 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:21:12.718972 (+ 341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:21:12.719487 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:21:12.720264 (+ 777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:21:12.720465 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:21:12.720550 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:21:12.723106 (+ 2556us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.51.130:53951: BlockingRecv error: recv got EOF from 127.21.51.130:53951 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
Aug 09 16:21:12 dist-test-slave-s11t krb5kdc[1930](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.51.131@KRBTEST.COM for oryx/127.21.51.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 16:21:12 dist-test-slave-s11t krb5kdc[1930](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.51.131@KRBTEST.COM for oryx/127.21.51.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 16:21:12.856729 3711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:21:12.851763 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:42871 (local address 127.21.51.131:60161)
0809 16:21:12.851931 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0809 16:21:12.851943 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 16:21:12.852009 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:21:12.852147 (+ 138us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:21:12.852148 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:21:12.852326 (+ 178us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:21:12.852377 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:21:12.852378 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:21:12.853192 (+ 814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:21:12.853193 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:21:12.853861 (+ 668us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:21:12.853864 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:21:12.854294 (+ 430us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:21:12.854295 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:21:12.854339 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:21:12.854880 (+ 541us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:21:12.854890 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:21:12.856625 (+ 1735us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.51.190:42871: Server oryx/127.21.51.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":92,"threads_started":1}
W20260809 16:21:12.856896 3707 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:21:12.851828 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.51.131:60161 (local address 127.21.51.190:42871)
0809 16:21:12.851892 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0809 16:21:12.851894 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:21:12.852014 (+ 120us) server_negotiation.cc:407] Connection header received
0809 16:21:12.852058 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:21:12.852059 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:21:12.852073 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:21:12.852127 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:21:12.852472 (+ 345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:21:12.853093 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:21:12.853924 (+ 831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:21:12.854219 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:21:12.854300 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:21:12.856832 (+ 2532us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.51.131:60161: BlockingRecv error: recv got EOF from 127.21.51.131:60161 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
Aug 09 16:21:13 dist-test-slave-s11t krb5kdc[1930](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.51.129@KRBTEST.COM for oryx/127.21.51.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 16:21:13 dist-test-slave-s11t krb5kdc[1930](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.51.129@KRBTEST.COM for oryx/127.21.51.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 16:21:13.605410 3712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:21:13.600358 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:42871 (local address 127.21.51.129:49287)
0809 16:21:13.600531 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0809 16:21:13.600542 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 16:21:13.600595 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:21:13.600749 (+ 154us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:21:13.600750 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:21:13.600922 (+ 172us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:21:13.600975 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:21:13.600977 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:21:13.601646 (+ 669us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:21:13.601647 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:21:13.602202 (+ 555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:21:13.602204 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:21:13.602569 (+ 365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:21:13.602570 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:21:13.602612 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:21:13.603067 (+ 455us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:21:13.603071 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:21:13.605249 (+ 2178us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.51.190:42871: Server oryx/127.21.51.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":104,"threads_started":1}
W20260809 16:21:13.605515 3713 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:21:13.600456 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.51.129:49287 (local address 127.21.51.190:42871)
0809 16:21:13.600589 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0809 16:21:13.600591 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:21:13.600600 (+ 9us) server_negotiation.cc:407] Connection header received
0809 16:21:13.600650 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:21:13.600650 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:21:13.600661 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:21:13.600731 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:21:13.601044 (+ 313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:21:13.601571 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:21:13.602298 (+ 727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:21:13.602499 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:21:13.602556 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:21:13.605399 (+ 2843us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.51.129:49287: BlockingRecv error: recv got EOF from 127.21.51.129:49287 (error 108)
Metrics: {"server-negotiator.queue_time_us":110,"thread_start_us":57,"threads_started":1}
Aug 09 16:21:13 dist-test-slave-s11t krb5kdc[1930](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.51.130@KRBTEST.COM for oryx/127.21.51.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 16:21:13 dist-test-slave-s11t krb5kdc[1930](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.51.130@KRBTEST.COM for oryx/127.21.51.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 16:21:13.728471 3714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:21:13.723579 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:42871 (local address 127.21.51.130:45517)
0809 16:21:13.723759 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0809 16:21:13.723770 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 16:21:13.723832 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:21:13.723970 (+ 138us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:21:13.723970 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:21:13.724139 (+ 169us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:21:13.724192 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:21:13.724193 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:21:13.724866 (+ 673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:21:13.724867 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:21:13.725465 (+ 598us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:21:13.725467 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:21:13.725841 (+ 374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:21:13.725842 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:21:13.725889 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:21:13.726458 (+ 569us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:21:13.726467 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:21:13.728413 (+ 1946us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.51.190:42871: Server oryx/127.21.51.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":125,"threads_started":1}
W20260809 16:21:13.728623 3713 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:21:13.723638 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.51.130:45517 (local address 127.21.51.190:42871)
0809 16:21:13.723706 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0809 16:21:13.723709 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 16:21:13.723830 (+ 121us) server_negotiation.cc:407] Connection header received
0809 16:21:13.723879 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:21:13.723880 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:21:13.723892 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:21:13.723948 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:21:13.724247 (+ 299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:21:13.724786 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:21:13.725550 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:21:13.725762 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:21:13.725823 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:21:13.728538 (+ 2715us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.51.130:45517: BlockingRecv error: recv got EOF from 127.21.51.130:45517 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
I20260809 16:21:13.776011 21710 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrNqUCn/build/release/bin/kudu with pid 2917
I20260809 16:21:13.788646 21710 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrNqUCn/build/release/bin/kudu with pid 3182
I20260809 16:21:13.799021 21710 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrNqUCn/build/release/bin/kudu with pid 3411
I20260809 16:21:13.813460 21710 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrNqUCn/build/release/bin/kudu with pid 2800
2026-08-09T16:21:13Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (17837 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-08-09T16:21:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T16:21:13Z Disabled control of system clock
I20260809 16:21:13.845319 21710 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrNqUCn/build/release/bin/kudu
/tmp/dist-test-taskrNqUCn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786292417722232-21710-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786292417722232-21710-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786292417722232-21710-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786292417722232-21710-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.51.190:45369
--webserver_interface=127.21.51.190
--webserver_port=0
--builtin_ntp_servers=127.21.51.148:34703
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.51.190:45369
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260809 16:21:13.928632 3719 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:21:13.928854 3719 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:21:13.928874 3719 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:21:13.929965 3719 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 16:21:13.929998 3719 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:21:13.930011 3719 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:21:13.930023 3719 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 16:21:13.930035 3719 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 16:21:13.931337 3719 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.51.148:34703
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786292417722232-21710-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786292417722232-21710-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.51.190:45369
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.51.190:45369
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786292417722232-21710-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.51.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3719
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786292417722232-21710-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:21:13.931535 3719 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:21:13.931777 3719 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:21:13.936731 3730 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:21:13.936739 3732 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:21:13.936808 3729 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:21:13.936839 3719 server_base.cc:1034] running on GCE node
I20260809 16:21:13.937202 3719 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:21:13.937577 3719 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:21:13.938748 3719 hybrid_clock.cc:648] HybridClock initialized: now 1786292473938735 us; error 43 us; skew 500 ppm
I20260809 16:21:13.940001 3719 webserver.cc:460] Webserver started at http://127.21.51.190:44153/ using document root <none> and password file <none>
I20260809 16:21:13.940254 3719 fs_manager.cc:359] Metadata directory not provided
I20260809 16:21:13.940325 3719 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:21:13.940512 3719 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:21:13.941638 3719 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786292417722232-21710-0/minicluster-data/master-0/data/instance:
uuid: "ef37606b72484e7a9b69f72af17404b0"
format_stamp: "Formatted at 2026-08-09 16:21:13 on dist-test-slave-s11t"
server_key: "5dae41be7da2853c23c1652426a35a34"
server_key_iv: "f4965e65cadb33f958381fc38c9d482c"
server_key_version: "encryptionkey@0"
I20260809 16:21:13.941999 3719 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786292417722232-21710-0/minicluster-data/master-0/wal/instance:
uuid: "ef37606b72484e7a9b69f72af17404b0"
format_stamp: "Formatted at 2026-08-09 16:21:13 on dist-test-slave-s11t"
server_key: "5dae41be7da2853c23c1652426a35a34"
server_key_iv: "f4965e65cadb33f958381fc38c9d482c"
server_key_version: "encryptionkey@0"
I20260809 16:21:13.943349 3719 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 16:21:13.944082 3741 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:21:13.944263 3719 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 16:21:13.944365 3719 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786292417722232-21710-0/minicluster-data/master-0/data,/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786292417722232-21710-0/minicluster-data/master-0/wal
uuid: "ef37606b72484e7a9b69f72af17404b0"
format_stamp: "Formatted at 2026-08-09 16:21:13 on dist-test-slave-s11t"
server_key: "5dae41be7da2853c23c1652426a35a34"
server_key_iv: "f4965e65cadb33f958381fc38c9d482c"
server_key_version: "encryptionkey@0"
I20260809 16:21:13.944459 3719 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786292417722232-21710-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786292417722232-21710-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786292417722232-21710-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:21:13.952307 3719 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:21:13.952590 3719 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:21:13.952777 3719 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:21:13.956908 3719 rpc_server.cc:307] RPC server started. Bound to: 127.21.51.190:45369
I20260809 16:21:13.956959 3822 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.51.190:45369 every 8 connection(s)
I20260809 16:21:13.957291 3719 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786292417722232-21710-0/minicluster-data/master-0/data/info.pb
I20260809 16:21:13.957981 3823 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:21:13.959815 21710 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrNqUCn/build/release/bin/kudu as pid 3719
I20260809 16:21:13.959950 21710 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786292417722232-21710-0/minicluster-data/master-0/wal/instance
I20260809 16:21:13.960146 21710 external_mini_cluster.cc:1428] Setting key 77846b945788af1609eb4f0e0c89701e
I20260809 16:21:13.960840 3823 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ef37606b72484e7a9b69f72af17404b0: Bootstrap starting.
I20260809 16:21:13.961591 3823 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ef37606b72484e7a9b69f72af17404b0: Neither blocks nor log segments found. Creating new log.
I20260809 16:21:13.962007 3823 log.cc:824] T 00000000000000000000000000000000 P ef37606b72484e7a9b69f72af17404b0: Log is configured to *not* fsync() on all Append() calls
I20260809 16:21:13.962824 3823 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ef37606b72484e7a9b69f72af17404b0: No bootstrap required, opened a new log
I20260809 16:21:13.964530 3823 raft_consensus.cc:348] T 00000000000000000000000000000000 P ef37606b72484e7a9b69f72af17404b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef37606b72484e7a9b69f72af17404b0" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 45369 } }
I20260809 16:21:13.964686 3823 raft_consensus.cc:374] T 00000000000000000000000000000000 P ef37606b72484e7a9b69f72af17404b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:21:13.964807 3823 raft_consensus.cc:729] T 00000000000000000000000000000000 P ef37606b72484e7a9b69f72af17404b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef37606b72484e7a9b69f72af17404b0, State: Initialized, Role: FOLLOWER
I20260809 16:21:13.964978 3823 consensus_queue.cc:260] T 00000000000000000000000000000000 P ef37606b72484e7a9b69f72af17404b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef37606b72484e7a9b69f72af17404b0" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 45369 } }
I20260809 16:21:13.965097 3823 raft_consensus.cc:388] T 00000000000000000000000000000000 P ef37606b72484e7a9b69f72af17404b0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 16:21:13.965132 3823 raft_consensus.cc:482] T 00000000000000000000000000000000 P ef37606b72484e7a9b69f72af17404b0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 16:21:13.965174 3823 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ef37606b72484e7a9b69f72af17404b0 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:21:13.966073 3823 raft_consensus.cc:504] T 00000000000000000000000000000000 P ef37606b72484e7a9b69f72af17404b0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef37606b72484e7a9b69f72af17404b0" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 45369 } }
I20260809 16:21:13.966195 3823 leader_election.cc:302] T 00000000000000000000000000000000 P ef37606b72484e7a9b69f72af17404b0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ef37606b72484e7a9b69f72af17404b0; no voters:
I20260809 16:21:13.966248 3827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:21:13.960715 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45562 (local address 127.21.51.190:45369)
0809 16:21:13.960945 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0809 16:21:13.960955 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 16:21:13.960994 (+ 39us) server_negotiation.cc:407] Connection header received
0809 16:21:13.961612 (+ 618us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:21:13.961620 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:21:13.961703 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:21:13.961873 (+ 170us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:21:13.962099 (+ 226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:21:13.963073 (+ 974us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:21:13.964194 (+ 1121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:21:13.964613 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:21:13.964802 (+ 189us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:21:13.965486 (+ 684us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:21:13.965490 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:21:13.965490 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 16:21:13.965699 (+ 209us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:21:13.965743 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 16:21:13.965758 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:21:13.965836 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0809 16:21:13.965901 (+ 65us) server_negotiation.cc:299] Negotiation successful
0809 16:21:13.965985 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":118,"thread_start_us":87,"threads_started":1}
I20260809 16:21:13.966348 3823 leader_election.cc:288] T 00000000000000000000000000000000 P ef37606b72484e7a9b69f72af17404b0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 16:21:13.966521 3831 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ef37606b72484e7a9b69f72af17404b0 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:21:13.966601 3823 sys_catalog.cc:564] T 00000000000000000000000000000000 P ef37606b72484e7a9b69f72af17404b0 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 16:21:13.966677 3831 raft_consensus.cc:686] T 00000000000000000000000000000000 P ef37606b72484e7a9b69f72af17404b0 [term 1 LEADER]: Becoming Leader. State: Replica: ef37606b72484e7a9b69f72af17404b0, State: Running, Role: LEADER
I20260809 16:21:13.966781 3831 consensus_queue.cc:237] T 00000000000000000000000000000000 P ef37606b72484e7a9b69f72af17404b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef37606b72484e7a9b69f72af17404b0" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 45369 } }
I20260809 16:21:13.967101 3831 sys_catalog.cc:455] T 00000000000000000000000000000000 P ef37606b72484e7a9b69f72af17404b0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ef37606b72484e7a9b69f72af17404b0. Latest consensus state: current_term: 1 leader_uuid: "ef37606b72484e7a9b69f72af17404b0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef37606b72484e7a9b69f72af17404b0" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 45369 } } }
I20260809 16:21:13.967166 3829 sys_catalog.cc:455] T 00000000000000000000000000000000 P ef37606b72484e7a9b69f72af17404b0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ef37606b72484e7a9b69f72af17404b0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef37606b72484e7a9b69f72af17404b0" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 45369 } } }
I20260809 16:21:13.967273 3831 sys_catalog.cc:458] T 00000000000000000000000000000000 P ef37606b72484e7a9b69f72af17404b0 [sys.catalog]: This master's current role is: LEADER
I20260809 16:21:13.967337 3829 sys_catalog.cc:458] T 00000000000000000000000000000000 P ef37606b72484e7a9b69f72af17404b0 [sys.catalog]: This master's current role is: LEADER
I20260809 16:21:13.967548 3844 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 16:21:13.968116 3844 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 16:21:13.969806 3844 catalog_manager.cc:1347] Generated new cluster ID: 2d9d3168e0134df8aa035de0a0914266
I20260809 16:21:13.969861 3844 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 16:21:13.993502 3844 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 16:21:13.994056 3844 catalog_manager.cc:1504] Loading token signing keys...
I20260809 16:21:14.002368 3844 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ef37606b72484e7a9b69f72af17404b0: Generated new TSK 0
I20260809 16:21:14.002566 3844 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 16:21:14.008668 21710 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrNqUCn/build/release/bin/kudu
/tmp/dist-test-taskrNqUCn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786292417722232-21710-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786292417722232-21710-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786292417722232-21710-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786292417722232-21710-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.51.129:0
--local_ip_for_outbound_sockets=127.21.51.129
--webserver_interface=127.21.51.129
--webserver_port=0
--tserver_master_addrs=127.21.51.190:45369
--builtin_ntp_servers=127.21.51.148:34703
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260809 16:21:14.111059 3857 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:21:14.111253 3857 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:21:14.111272 3857 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:21:14.112396 3857 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:21:14.112434 3857 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:21:14.112458 3857 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.51.129
I20260809 16:21:14.114342 3857 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.51.148:34703
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786292417722232-21710-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786292417722232-21710-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.51.129:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786292417722232-21710-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.51.129
--webserver_port=0
--tserver_master_addrs=127.21.51.190:45369
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3857
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.51.129
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786292417722232-21710-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:21:14.114570 3857 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:21:14.114874 3857 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:21:14.119993 3864 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:21:14.120141 3857 server_base.cc:1034] running on GCE node
W20260809 16:21:14.120260 3870 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:21:14.119998 3865 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:21:14.120572 3857 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:21:14.120849 3857 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:21:14.122020 3857 hybrid_clock.cc:648] HybridClock initialized: now 1786292474122007 us; error 29 us; skew 500 ppm
I20260809 16:21:14.123363 3857 webserver.cc:460] Webserver started at http://127.21.51.129:36855/ using document root <none> and password file <none>
I20260809 16:21:14.123631 3857 fs_manager.cc:359] Metadata directory not provided
I20260809 16:21:14.123708 3857 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:21:14.123863 3857 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:21:14.125067 3857 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786292417722232-21710-0/minicluster-data/ts-0/data/instance:
uuid: "c60aac4f20ad463293d112045626c18f"
format_stamp: "Formatted at 2026-08-09 16:21:14 on dist-test-slave-s11t"
server_key: "4289b74acad0bda7700c8b8a434aa7b4"
server_key_iv: "810323e7c38278e2830adef11e1d253d"
server_key_version: "encryptionkey@0"
I20260809 16:21:14.125433 3857 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786292417722232-21710-0/minicluster-data/ts-0/wal/instance:
uuid: "c60aac4f20ad463293d112045626c18f"
format_stamp: "Formatted at 2026-08-09 16:21:14 on dist-test-slave-s11t"
server_key: "4289b74acad0bda7700c8b8a434aa7b4"
server_key_iv: "810323e7c38278e2830adef11e1d253d"
server_key_version: "encryptionkey@0"
I20260809 16:21:14.126701 3857 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:21:14.127393 3879 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:21:14.127566 3857 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 16:21:14.127657 3857 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786292417722232-21710-0/minicluster-data/ts-0/data,/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786292417722232-21710-0/minicluster-data/ts-0/wal
uuid: "c60aac4f20ad463293d112045626c18f"
format_stamp: "Formatted at 2026-08-09 16:21:14 on dist-test-slave-s11t"
server_key: "4289b74acad0bda7700c8b8a434aa7b4"
server_key_iv: "810323e7c38278e2830adef11e1d253d"
server_key_version: "encryptionkey@0"
I20260809 16:21:14.127740 3857 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786292417722232-21710-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786292417722232-21710-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786292417722232-21710-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:21:14.141848 3857 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:21:14.142132 3857 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:21:14.142282 3857 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:21:14.142509 3857 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:21:14.142841 3857 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:21:14.142904 3857 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:21:14.142946 3857 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:21:14.142974 3857 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:21:14.148914 3857 rpc_server.cc:307] RPC server started. Bound to: 127.21.51.129:46171
I20260809 16:21:14.148950 4054 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.51.129:46171 every 8 connection(s)
I20260809 16:21:14.149298 3857 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786292417722232-21710-0/minicluster-data/ts-0/data/info.pb
I20260809 16:21:14.154249 3827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:21:14.150075 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.51.129:43049 (local address 127.21.51.190:45369)
0809 16:21:14.150131 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0809 16:21:14.150133 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:21:14.150361 (+ 228us) server_negotiation.cc:407] Connection header received
0809 16:21:14.150822 (+ 461us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:21:14.150823 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:21:14.150837 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:21:14.150899 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:21:14.151500 (+ 601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:21:14.152008 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:21:14.153057 (+ 1049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:21:14.153254 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:21:14.153310 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:21:14.154011 (+ 701us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:21:14.154013 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:21:14.154013 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 16:21:14.154017 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:21:14.154030 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 16:21:14.154033 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:21:14.154077 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0809 16:21:14.154157 (+ 80us) server_negotiation.cc:299] Negotiation successful
0809 16:21:14.154183 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260809 16:21:14.154424 4062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:21:14.150061 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:45369 (local address 127.21.51.129:43049)
0809 16:21:14.150237 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0809 16:21:14.150276 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0809 16:21:14.150739 (+ 463us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:21:14.150951 (+ 212us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:21:14.150955 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:21:14.150962 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:21:14.151437 (+ 475us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:21:14.151440 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:21:14.152090 (+ 650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:21:14.152090 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:21:14.152986 (+ 896us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:21:14.152989 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:21:14.153327 (+ 338us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:21:14.153327 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:21:14.153430 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:21:14.153875 (+ 445us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 16:21:14.153875 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:21:14.153914 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 16:21:14.153934 (+ 20us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:21:14.154089 (+ 155us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:21:14.154094 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:21:14.154096 (+ 2us) client_negotiation.cc:769] Sending connection context
0809 16:21:14.154145 (+ 49us) client_negotiation.cc:240] Negotiation successful
0809 16:21:14.154218 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":81,"threads_started":1}
I20260809 16:21:14.154946 4057 heartbeater.cc:347] Connected to a master server at 127.21.51.190:45369
I20260809 16:21:14.155046 4057 heartbeater.cc:464] Registering TS with master...
I20260809 16:21:14.155095 21710 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrNqUCn/build/release/bin/kudu as pid 3857
I20260809 16:21:14.155184 21710 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786292417722232-21710-0/minicluster-data/ts-0/wal/instance
I20260809 16:21:14.155243 4057 heartbeater.cc:511] Master 127.21.51.190:45369 requested a full tablet report, sending...
I20260809 16:21:14.155392 21710 external_mini_cluster.cc:1428] Setting key 68a39d60e0fa978d5a26a1a069608d9e
I20260809 16:21:14.155792 3764 ts_manager.cc:194] Registered new tserver with Master: c60aac4f20ad463293d112045626c18f (127.21.51.129:46171)
I20260809 16:21:14.156490 21710 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrNqUCn/build/release/bin/kudu
/tmp/dist-test-taskrNqUCn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786292417722232-21710-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786292417722232-21710-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786292417722232-21710-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786292417722232-21710-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.51.130:0
--local_ip_for_outbound_sockets=127.21.51.130
--webserver_interface=127.21.51.130
--webserver_port=0
--tserver_master_addrs=127.21.51.190:45369
--builtin_ntp_servers=127.21.51.148:34703
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260809 16:21:14.156523 3764 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.21.51.129:43049
W20260809 16:21:14.238011 4063 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:21:14.238215 4063 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:21:14.238237 4063 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:21:14.239380 4063 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:21:14.239423 4063 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:21:14.239447 4063 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.51.130
I20260809 16:21:14.240820 4063 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.51.148:34703
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786292417722232-21710-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786292417722232-21710-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.51.130:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786292417722232-21710-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.51.130
--webserver_port=0
--tserver_master_addrs=127.21.51.190:45369
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4063
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.51.130
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786292417722232-21710-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:21:14.241070 4063 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:21:14.241382 4063 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:21:14.246829 4075 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:21:14.247035 4063 server_base.cc:1034] running on GCE node
W20260809 16:21:14.246825 4076 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:21:14.247174 4079 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:21:14.247381 4063 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:21:14.247643 4063 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:21:14.248826 4063 hybrid_clock.cc:648] HybridClock initialized: now 1786292474248805 us; error 39 us; skew 500 ppm
I20260809 16:21:14.250404 4063 webserver.cc:460] Webserver started at http://127.21.51.130:34347/ using document root <none> and password file <none>
I20260809 16:21:14.250707 4063 fs_manager.cc:359] Metadata directory not provided
I20260809 16:21:14.250777 4063 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:21:14.250919 4063 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:21:14.252409 4063 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786292417722232-21710-0/minicluster-data/ts-1/data/instance:
uuid: "06e23de9a316423983693c81380bed45"
format_stamp: "Formatted at 2026-08-09 16:21:14 on dist-test-slave-s11t"
server_key: "ed9aec3de44bec47e62fa268a4c490e3"
server_key_iv: "4da4d54a5f8303befdf45297a07ce747"
server_key_version: "encryptionkey@0"
I20260809 16:21:14.252815 4063 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786292417722232-21710-0/minicluster-data/ts-1/wal/instance:
uuid: "06e23de9a316423983693c81380bed45"
format_stamp: "Formatted at 2026-08-09 16:21:14 on dist-test-slave-s11t"
server_key: "ed9aec3de44bec47e62fa268a4c490e3"
server_key_iv: "4da4d54a5f8303befdf45297a07ce747"
server_key_version: "encryptionkey@0"
I20260809 16:21:14.254118 4063 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:21:14.254904 4085 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:21:14.255090 4063 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:21:14.255187 4063 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786292417722232-21710-0/minicluster-data/ts-1/data,/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786292417722232-21710-0/minicluster-data/ts-1/wal
uuid: "06e23de9a316423983693c81380bed45"
format_stamp: "Formatted at 2026-08-09 16:21:14 on dist-test-slave-s11t"
server_key: "ed9aec3de44bec47e62fa268a4c490e3"
server_key_iv: "4da4d54a5f8303befdf45297a07ce747"
server_key_version: "encryptionkey@0"
I20260809 16:21:14.255273 4063 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786292417722232-21710-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786292417722232-21710-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786292417722232-21710-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:21:14.279508 4063 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:21:14.279826 4063 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:21:14.279989 4063 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:21:14.280221 4063 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:21:14.280550 4063 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:21:14.280603 4063 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:21:14.280646 4063 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:21:14.280674 4063 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:21:14.286561 4063 rpc_server.cc:307] RPC server started. Bound to: 127.21.51.130:43657
I20260809 16:21:14.286618 4268 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.51.130:43657 every 8 connection(s)
I20260809 16:21:14.286979 4063 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786292417722232-21710-0/minicluster-data/ts-1/data/info.pb
I20260809 16:21:14.292025 3827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:21:14.287745 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.51.130:60175 (local address 127.21.51.190:45369)
0809 16:21:14.287792 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0809 16:21:14.287794 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:21:14.288009 (+ 215us) server_negotiation.cc:407] Connection header received
0809 16:21:14.288510 (+ 501us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:21:14.288511 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:21:14.288521 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:21:14.288570 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:21:14.289290 (+ 720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:21:14.289780 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:21:14.290780 (+ 1000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:21:14.290959 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:21:14.291008 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:21:14.291795 (+ 787us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:21:14.291796 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:21:14.291797 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 16:21:14.291801 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:21:14.291813 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 16:21:14.291817 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:21:14.291859 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0809 16:21:14.291945 (+ 86us) server_negotiation.cc:299] Negotiation successful
0809 16:21:14.291965 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260809 16:21:14.292290 4271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:21:14.287715 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:45369 (local address 127.21.51.130:60175)
0809 16:21:14.287899 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0809 16:21:14.287934 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0809 16:21:14.288423 (+ 489us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:21:14.288630 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:21:14.288633 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:21:14.288642 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:21:14.289210 (+ 568us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:21:14.289216 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:21:14.289877 (+ 661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:21:14.289878 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:21:14.290691 (+ 813us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:21:14.290702 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:21:14.291035 (+ 333us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:21:14.291036 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:21:14.291146 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:21:14.291630 (+ 484us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 16:21:14.291631 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:21:14.291659 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 16:21:14.291680 (+ 21us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:21:14.291873 (+ 193us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:21:14.291877 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:21:14.291879 (+ 2us) client_negotiation.cc:769] Sending connection context
0809 16:21:14.291931 (+ 52us) client_negotiation.cc:240] Negotiation successful
0809 16:21:14.292012 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":92,"threads_started":1}
I20260809 16:21:14.292336 21710 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrNqUCn/build/release/bin/kudu as pid 4063
I20260809 16:21:14.292430 21710 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786292417722232-21710-0/minicluster-data/ts-1/wal/instance
I20260809 16:21:14.292667 21710 external_mini_cluster.cc:1428] Setting key c7b0c617ce61c66dcc0588428eeebac9
I20260809 16:21:14.292889 4269 heartbeater.cc:347] Connected to a master server at 127.21.51.190:45369
I20260809 16:21:14.292984 4269 heartbeater.cc:464] Registering TS with master...
I20260809 16:21:14.293181 4269 heartbeater.cc:511] Master 127.21.51.190:45369 requested a full tablet report, sending...
I20260809 16:21:14.293627 3764 ts_manager.cc:194] Registered new tserver with Master: 06e23de9a316423983693c81380bed45 (127.21.51.130:43657)
I20260809 16:21:14.293642 21710 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrNqUCn/build/release/bin/kudu
/tmp/dist-test-taskrNqUCn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786292417722232-21710-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786292417722232-21710-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786292417722232-21710-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786292417722232-21710-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.51.131:0
--local_ip_for_outbound_sockets=127.21.51.131
--webserver_interface=127.21.51.131
--webserver_port=0
--tserver_master_addrs=127.21.51.190:45369
--builtin_ntp_servers=127.21.51.148:34703
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260809 16:21:14.294095 3764 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.21.51.130:60175
W20260809 16:21:14.376871 4272 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:21:14.377071 4272 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:21:14.377092 4272 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:21:14.378219 4272 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:21:14.378259 4272 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:21:14.378283 4272 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.51.131
I20260809 16:21:14.379650 4272 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.51.148:34703
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786292417722232-21710-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786292417722232-21710-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.51.131:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786292417722232-21710-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.51.131
--webserver_port=0
--tserver_master_addrs=127.21.51.190:45369
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4272
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.51.131
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786292417722232-21710-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:21:14.379833 4272 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:21:14.380085 4272 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:21:14.384970 4283 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:21:14.384984 4280 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:21:14.385160 4272 server_base.cc:1034] running on GCE node
W20260809 16:21:14.384985 4281 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:21:14.385437 4272 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:21:14.385782 4272 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:21:14.386981 4272 hybrid_clock.cc:648] HybridClock initialized: now 1786292474386970 us; error 36 us; skew 500 ppm
I20260809 16:21:14.388276 4272 webserver.cc:460] Webserver started at http://127.21.51.131:34923/ using document root <none> and password file <none>
I20260809 16:21:14.388533 4272 fs_manager.cc:359] Metadata directory not provided
I20260809 16:21:14.388609 4272 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:21:14.388798 4272 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:21:14.389891 4272 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786292417722232-21710-0/minicluster-data/ts-2/data/instance:
uuid: "1170187a4bca428eb36b75afd8975b21"
format_stamp: "Formatted at 2026-08-09 16:21:14 on dist-test-slave-s11t"
server_key: "d77f3656404761d2cfaf032bab17984a"
server_key_iv: "e72d6ed48e3893ec178d1108319fe96a"
server_key_version: "encryptionkey@0"
I20260809 16:21:14.390264 4272 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786292417722232-21710-0/minicluster-data/ts-2/wal/instance:
uuid: "1170187a4bca428eb36b75afd8975b21"
format_stamp: "Formatted at 2026-08-09 16:21:14 on dist-test-slave-s11t"
server_key: "d77f3656404761d2cfaf032bab17984a"
server_key_iv: "e72d6ed48e3893ec178d1108319fe96a"
server_key_version: "encryptionkey@0"
I20260809 16:21:14.391613 4272 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 16:21:14.392320 4295 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:21:14.392485 4272 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 16:21:14.392592 4272 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786292417722232-21710-0/minicluster-data/ts-2/data,/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786292417722232-21710-0/minicluster-data/ts-2/wal
uuid: "1170187a4bca428eb36b75afd8975b21"
format_stamp: "Formatted at 2026-08-09 16:21:14 on dist-test-slave-s11t"
server_key: "d77f3656404761d2cfaf032bab17984a"
server_key_iv: "e72d6ed48e3893ec178d1108319fe96a"
server_key_version: "encryptionkey@0"
I20260809 16:21:14.392668 4272 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786292417722232-21710-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786292417722232-21710-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786292417722232-21710-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:21:14.417712 4272 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:21:14.418054 4272 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:21:14.418216 4272 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:21:14.418467 4272 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:21:14.418854 4272 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:21:14.418915 4272 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:21:14.418952 4272 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:21:14.418979 4272 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:21:14.424770 4272 rpc_server.cc:307] RPC server started. Bound to: 127.21.51.131:43391
I20260809 16:21:14.424862 4470 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.51.131:43391 every 8 connection(s)
I20260809 16:21:14.425191 4272 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786292417722232-21710-0/minicluster-data/ts-2/data/info.pb
I20260809 16:21:14.429171 21710 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrNqUCn/build/release/bin/kudu as pid 4272
I20260809 16:21:14.429289 21710 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786292417722232-21710-0/minicluster-data/ts-2/wal/instance
I20260809 16:21:14.429518 21710 external_mini_cluster.cc:1428] Setting key fd551c7c6a6d4bf8e5852901813db260
I20260809 16:21:14.431026 3827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:21:14.426005 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.51.131:34485 (local address 127.21.51.190:45369)
0809 16:21:14.426074 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0809 16:21:14.426076 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:21:14.426332 (+ 256us) server_negotiation.cc:407] Connection header received
0809 16:21:14.426796 (+ 464us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:21:14.426797 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:21:14.426806 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:21:14.426870 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:21:14.427579 (+ 709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:21:14.428072 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:21:14.429265 (+ 1193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:21:14.429530 (+ 265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:21:14.429621 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:21:14.430815 (+ 1194us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:21:14.430816 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:21:14.430817 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 16:21:14.430821 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:21:14.430840 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 16:21:14.430844 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:21:14.430887 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0809 16:21:14.430946 (+ 59us) server_negotiation.cc:299] Negotiation successful
0809 16:21:14.430968 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260809 16:21:14.431253 4474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:21:14.425972 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:45369 (local address 127.21.51.131:34485)
0809 16:21:14.426165 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0809 16:21:14.426205 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0809 16:21:14.426708 (+ 503us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:21:14.426927 (+ 219us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:21:14.426930 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:21:14.426946 (+ 16us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:21:14.427442 (+ 496us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:21:14.427446 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:21:14.428162 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:21:14.428163 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:21:14.429156 (+ 993us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:21:14.429159 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:21:14.429797 (+ 638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:21:14.429798 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:21:14.429915 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:21:14.430550 (+ 635us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 16:21:14.430551 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:21:14.430595 (+ 44us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 16:21:14.430641 (+ 46us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:21:14.430889 (+ 248us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:21:14.430892 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:21:14.430894 (+ 2us) client_negotiation.cc:769] Sending connection context
0809 16:21:14.430942 (+ 48us) client_negotiation.cc:240] Negotiation successful
0809 16:21:14.431029 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":91,"threads_started":1}
I20260809 16:21:14.431697 4472 heartbeater.cc:347] Connected to a master server at 127.21.51.190:45369
I20260809 16:21:14.431794 4472 heartbeater.cc:464] Registering TS with master...
I20260809 16:21:14.431949 4472 heartbeater.cc:511] Master 127.21.51.190:45369 requested a full tablet report, sending...
I20260809 16:21:14.432304 3764 ts_manager.cc:194] Registered new tserver with Master: 1170187a4bca428eb36b75afd8975b21 (127.21.51.131:43391)
I20260809 16:21:14.432734 3764 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.21.51.131:34485
I20260809 16:21:14.441363 21710 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260809 16:21:14.443420 3827 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:21:14.442818 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34546 (local address 127.21.51.190:45369)
0809 16:21:14.442907 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0809 16:21:14.442909 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:21:14.443078 (+ 169us) server_negotiation.cc:407] Connection header received
0809 16:21:14.443125 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:21:14.443125 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:21:14.443135 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:21:14.443189 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:21:14.443355 (+ 166us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34546: BlockingRecv error: recv got EOF from 127.0.0.1:34546 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
I20260809 16:21:14.443490 21710 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrNqUCn/build/release/bin/kudu with pid 3857
I20260809 16:21:14.451308 21710 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrNqUCn/build/release/bin/kudu with pid 4063
I20260809 16:21:14.463323 21710 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrNqUCn/build/release/bin/kudu with pid 4272
I20260809 16:21:14.475155 21710 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrNqUCn/build/release/bin/kudu with pid 3719
2026-08-09T16:21:14Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (660 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-08-09T16:21:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T16:21:14Z Disabled control of system clock
I20260809 16:21:14.505628 21710 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrNqUCn/build/release/bin/kudu
/tmp/dist-test-taskrNqUCn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786292417722232-21710-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786292417722232-21710-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786292417722232-21710-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786292417722232-21710-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.51.190:45221
--webserver_interface=127.21.51.190
--webserver_port=0
--builtin_ntp_servers=127.21.51.148:41717
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.51.190:45221
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260809 16:21:14.588899 4485 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:21:14.589107 4485 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:21:14.589128 4485 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:21:14.590217 4485 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 16:21:14.590255 4485 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:21:14.590266 4485 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:21:14.590279 4485 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 16:21:14.590291 4485 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 16:21:14.591585 4485 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.51.148:41717
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786292417722232-21710-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786292417722232-21710-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.51.190:45221
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.51.190:45221
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786292417722232-21710-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.51.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4485
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786292417722232-21710-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:21:14.591782 4485 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:21:14.592036 4485 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:21:14.596975 4493 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:21:14.596997 4490 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:21:14.596973 4491 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:21:14.597220 4485 server_base.cc:1034] running on GCE node
I20260809 16:21:14.597498 4485 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:21:14.597851 4485 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:21:14.599014 4485 hybrid_clock.cc:648] HybridClock initialized: now 1786292474599011 us; error 38 us; skew 500 ppm
I20260809 16:21:14.600359 4485 webserver.cc:460] Webserver started at http://127.21.51.190:38933/ using document root <none> and password file <none>
I20260809 16:21:14.600633 4485 fs_manager.cc:359] Metadata directory not provided
I20260809 16:21:14.600730 4485 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:21:14.600899 4485 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:21:14.601996 4485 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786292417722232-21710-0/minicluster-data/master-0/data/instance:
uuid: "b0513dace06b4c29bfc8f33008d38562"
format_stamp: "Formatted at 2026-08-09 16:21:14 on dist-test-slave-s11t"
server_key: "1311d66517547a9363ddd839d4e07ce9"
server_key_iv: "1c241a46aafbc11ff4e71826a9bebdd7"
server_key_version: "encryptionkey@0"
I20260809 16:21:14.602367 4485 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786292417722232-21710-0/minicluster-data/master-0/wal/instance:
uuid: "b0513dace06b4c29bfc8f33008d38562"
format_stamp: "Formatted at 2026-08-09 16:21:14 on dist-test-slave-s11t"
server_key: "1311d66517547a9363ddd839d4e07ce9"
server_key_iv: "1c241a46aafbc11ff4e71826a9bebdd7"
server_key_version: "encryptionkey@0"
I20260809 16:21:14.603761 4485 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 16:21:14.604488 4501 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:21:14.604666 4485 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 16:21:14.604797 4485 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786292417722232-21710-0/minicluster-data/master-0/data,/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786292417722232-21710-0/minicluster-data/master-0/wal
uuid: "b0513dace06b4c29bfc8f33008d38562"
format_stamp: "Formatted at 2026-08-09 16:21:14 on dist-test-slave-s11t"
server_key: "1311d66517547a9363ddd839d4e07ce9"
server_key_iv: "1c241a46aafbc11ff4e71826a9bebdd7"
server_key_version: "encryptionkey@0"
I20260809 16:21:14.604887 4485 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786292417722232-21710-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786292417722232-21710-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786292417722232-21710-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:21:14.608289 4485 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:21:14.608560 4485 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:21:14.608745 4485 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:21:14.612637 4485 rpc_server.cc:307] RPC server started. Bound to: 127.21.51.190:45221
I20260809 16:21:14.612671 4600 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.51.190:45221 every 8 connection(s)
I20260809 16:21:14.613049 4485 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786292417722232-21710-0/minicluster-data/master-0/data/info.pb
I20260809 16:21:14.613726 4604 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:21:14.616415 4604 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b0513dace06b4c29bfc8f33008d38562: Bootstrap starting.
I20260809 16:21:14.617038 4604 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b0513dace06b4c29bfc8f33008d38562: Neither blocks nor log segments found. Creating new log.
I20260809 16:21:14.617292 4604 log.cc:824] T 00000000000000000000000000000000 P b0513dace06b4c29bfc8f33008d38562: Log is configured to *not* fsync() on all Append() calls
I20260809 16:21:14.617987 4604 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b0513dace06b4c29bfc8f33008d38562: No bootstrap required, opened a new log
I20260809 16:21:14.620078 21710 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrNqUCn/build/release/bin/kudu as pid 4485
I20260809 16:21:14.620008 4604 raft_consensus.cc:348] T 00000000000000000000000000000000 P b0513dace06b4c29bfc8f33008d38562 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b0513dace06b4c29bfc8f33008d38562" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 45221 } }
I20260809 16:21:14.620141 4604 raft_consensus.cc:374] T 00000000000000000000000000000000 P b0513dace06b4c29bfc8f33008d38562 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:21:14.620191 21710 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786292417722232-21710-0/minicluster-data/master-0/wal/instance
I20260809 16:21:14.620163 4604 raft_consensus.cc:729] T 00000000000000000000000000000000 P b0513dace06b4c29bfc8f33008d38562 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b0513dace06b4c29bfc8f33008d38562, State: Initialized, Role: FOLLOWER
I20260809 16:21:14.620335 4604 consensus_queue.cc:260] T 00000000000000000000000000000000 P b0513dace06b4c29bfc8f33008d38562 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b0513dace06b4c29bfc8f33008d38562" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 45221 } }
I20260809 16:21:14.620416 21710 external_mini_cluster.cc:1428] Setting key 393bfc4f3d7e50b949f7f213feca56c3
I20260809 16:21:14.620421 4604 raft_consensus.cc:388] T 00000000000000000000000000000000 P b0513dace06b4c29bfc8f33008d38562 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 16:21:14.620468 4604 raft_consensus.cc:482] T 00000000000000000000000000000000 P b0513dace06b4c29bfc8f33008d38562 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 16:21:14.620518 4604 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b0513dace06b4c29bfc8f33008d38562 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:21:14.621380 4604 raft_consensus.cc:504] T 00000000000000000000000000000000 P b0513dace06b4c29bfc8f33008d38562 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b0513dace06b4c29bfc8f33008d38562" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 45221 } }
I20260809 16:21:14.621501 4604 leader_election.cc:302] T 00000000000000000000000000000000 P b0513dace06b4c29bfc8f33008d38562 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b0513dace06b4c29bfc8f33008d38562; no voters:
I20260809 16:21:14.621711 4604 leader_election.cc:288] T 00000000000000000000000000000000 P b0513dace06b4c29bfc8f33008d38562 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 16:21:14.621773 4610 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b0513dace06b4c29bfc8f33008d38562 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:21:14.621994 4610 raft_consensus.cc:686] T 00000000000000000000000000000000 P b0513dace06b4c29bfc8f33008d38562 [term 1 LEADER]: Becoming Leader. State: Replica: b0513dace06b4c29bfc8f33008d38562, State: Running, Role: LEADER
I20260809 16:21:14.622033 4604 sys_catalog.cc:564] T 00000000000000000000000000000000 P b0513dace06b4c29bfc8f33008d38562 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 16:21:14.622124 4610 consensus_queue.cc:237] T 00000000000000000000000000000000 P b0513dace06b4c29bfc8f33008d38562 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b0513dace06b4c29bfc8f33008d38562" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 45221 } }
I20260809 16:21:14.622506 4610 sys_catalog.cc:455] T 00000000000000000000000000000000 P b0513dace06b4c29bfc8f33008d38562 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b0513dace06b4c29bfc8f33008d38562. Latest consensus state: current_term: 1 leader_uuid: "b0513dace06b4c29bfc8f33008d38562" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b0513dace06b4c29bfc8f33008d38562" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 45221 } } }
I20260809 16:21:14.622628 4610 sys_catalog.cc:458] T 00000000000000000000000000000000 P b0513dace06b4c29bfc8f33008d38562 [sys.catalog]: This master's current role is: LEADER
I20260809 16:21:14.622849 4611 sys_catalog.cc:455] T 00000000000000000000000000000000 P b0513dace06b4c29bfc8f33008d38562 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b0513dace06b4c29bfc8f33008d38562" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b0513dace06b4c29bfc8f33008d38562" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 45221 } } }
I20260809 16:21:14.622967 4611 sys_catalog.cc:458] T 00000000000000000000000000000000 P b0513dace06b4c29bfc8f33008d38562 [sys.catalog]: This master's current role is: LEADER
I20260809 16:21:14.623055 4609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:21:14.620986 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52282 (local address 127.21.51.190:45221)
0809 16:21:14.621214 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0809 16:21:14.621222 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 16:21:14.621248 (+ 26us) server_negotiation.cc:407] Connection header received
0809 16:21:14.621344 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:21:14.621349 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:21:14.621382 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:21:14.621538 (+ 156us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:21:14.622352 (+ 814us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:21:14.622355 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:21:14.622355 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 16:21:14.622359 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:21:14.622401 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 16:21:14.622418 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:21:14.622737 (+ 319us) server_negotiation.cc:1035] Waiting for connection context
0809 16:21:14.622773 (+ 36us) server_negotiation.cc:299] Negotiation successful
0809 16:21:14.622834 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":147,"thread_start_us":113,"threads_started":1}
W20260809 16:21:14.624166 4634 catalog_manager.cc:1558] T 00000000000000000000000000000000 P b0513dace06b4c29bfc8f33008d38562: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 16:21:14.624212 4634 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 16:21:14.624276 4618 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 16:21:14.624471 4618 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 16:21:14.626044 4618 catalog_manager.cc:1347] Generated new cluster ID: c988336d9e974087acb725acf49a0c01
I20260809 16:21:14.626101 4618 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 16:21:14.649794 4618 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 16:21:14.650014 4618 catalog_manager.cc:1504] Loading token signing keys...
I20260809 16:21:14.655910 4618 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b0513dace06b4c29bfc8f33008d38562: Generated new TSK 0
I20260809 16:21:14.656098 4618 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 16:21:14.665400 21710 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrNqUCn/build/release/bin/kudu
/tmp/dist-test-taskrNqUCn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786292417722232-21710-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786292417722232-21710-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786292417722232-21710-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786292417722232-21710-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.51.129:0
--local_ip_for_outbound_sockets=127.21.51.129
--webserver_interface=127.21.51.129
--webserver_port=0
--tserver_master_addrs=127.21.51.190:45221
--builtin_ntp_servers=127.21.51.148:41717
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260809 16:21:14.749171 4637 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:21:14.749413 4637 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:21:14.749454 4637 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:21:14.750710 4637 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:21:14.750772 4637 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:21:14.750811 4637 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.51.129
I20260809 16:21:14.752172 4637 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.51.148:41717
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786292417722232-21710-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786292417722232-21710-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.51.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786292417722232-21710-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.51.129
--webserver_port=0
--tserver_master_addrs=127.21.51.190:45221
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4637
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.51.129
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786292417722232-21710-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:21:14.752445 4637 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:21:14.752779 4637 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:21:14.757792 4643 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:21:14.757851 4649 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:21:14.757823 4645 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:21:14.758137 4637 server_base.cc:1034] running on GCE node
I20260809 16:21:14.758370 4637 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:21:14.758625 4637 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:21:14.759763 4637 hybrid_clock.cc:648] HybridClock initialized: now 1786292474759752 us; error 27 us; skew 500 ppm
I20260809 16:21:14.761065 4637 webserver.cc:460] Webserver started at http://127.21.51.129:39671/ using document root <none> and password file <none>
I20260809 16:21:14.761299 4637 fs_manager.cc:359] Metadata directory not provided
I20260809 16:21:14.761345 4637 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:21:14.761494 4637 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:21:14.762553 4637 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786292417722232-21710-0/minicluster-data/ts-0/data/instance:
uuid: "ad0ab468d44640338c156969ceb9cda4"
format_stamp: "Formatted at 2026-08-09 16:21:14 on dist-test-slave-s11t"
server_key: "0c8a5767209c0221da6106306b08caa2"
server_key_iv: "dde537fb5c12e227b55417675cff4603"
server_key_version: "encryptionkey@0"
I20260809 16:21:14.762933 4637 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786292417722232-21710-0/minicluster-data/ts-0/wal/instance:
uuid: "ad0ab468d44640338c156969ceb9cda4"
format_stamp: "Formatted at 2026-08-09 16:21:14 on dist-test-slave-s11t"
server_key: "0c8a5767209c0221da6106306b08caa2"
server_key_iv: "dde537fb5c12e227b55417675cff4603"
server_key_version: "encryptionkey@0"
I20260809 16:21:14.764820 4637 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.004s sys 0.000s
I20260809 16:21:14.765594 4656 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:21:14.765784 4637 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 16:21:14.765856 4637 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786292417722232-21710-0/minicluster-data/ts-0/data,/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786292417722232-21710-0/minicluster-data/ts-0/wal
uuid: "ad0ab468d44640338c156969ceb9cda4"
format_stamp: "Formatted at 2026-08-09 16:21:14 on dist-test-slave-s11t"
server_key: "0c8a5767209c0221da6106306b08caa2"
server_key_iv: "dde537fb5c12e227b55417675cff4603"
server_key_version: "encryptionkey@0"
I20260809 16:21:14.765964 4637 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786292417722232-21710-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786292417722232-21710-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786292417722232-21710-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:21:14.769349 4637 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:21:14.769603 4637 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:21:14.769739 4637 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:21:14.769948 4637 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:21:14.770248 4637 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:21:14.770283 4637 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:21:14.770304 4637 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:21:14.770349 4637 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:21:14.776114 4637 rpc_server.cc:307] RPC server started. Bound to: 127.21.51.129:39153
I20260809 16:21:14.776161 4818 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.51.129:39153 every 8 connection(s)
I20260809 16:21:14.776465 4637 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786292417722232-21710-0/minicluster-data/ts-0/data/info.pb
I20260809 16:21:14.779505 4609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:21:14.777190 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.51.129:35089 (local address 127.21.51.190:45221)
0809 16:21:14.777235 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0809 16:21:14.777237 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:21:14.777480 (+ 243us) server_negotiation.cc:407] Connection header received
0809 16:21:14.777978 (+ 498us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:21:14.777980 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:21:14.777988 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:21:14.778053 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:21:14.779258 (+ 1205us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:21:14.779260 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:21:14.779260 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 16:21:14.779265 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:21:14.779283 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 16:21:14.779287 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:21:14.779350 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0809 16:21:14.779417 (+ 67us) server_negotiation.cc:299] Negotiation successful
0809 16:21:14.779433 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260809 16:21:14.779698 4822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:21:14.777159 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:45221 (local address 127.21.51.129:35089)
0809 16:21:14.777366 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0809 16:21:14.777397 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0809 16:21:14.777868 (+ 471us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:21:14.778119 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:21:14.778123 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:21:14.778132 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:21:14.779104 (+ 972us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 16:21:14.779104 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:21:14.779133 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 16:21:14.779160 (+ 27us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:21:14.779338 (+ 178us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:21:14.779342 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:21:14.779345 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 16:21:14.779423 (+ 78us) client_negotiation.cc:240] Negotiation successful
0809 16:21:14.779485 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":94,"threads_started":1}
I20260809 16:21:14.780195 4819 heartbeater.cc:347] Connected to a master server at 127.21.51.190:45221
I20260809 16:21:14.780270 4819 heartbeater.cc:464] Registering TS with master...
I20260809 16:21:14.780436 4819 heartbeater.cc:511] Master 127.21.51.190:45221 requested a full tablet report, sending...
I20260809 16:21:14.781009 4540 ts_manager.cc:194] Registered new tserver with Master: ad0ab468d44640338c156969ceb9cda4 (127.21.51.129:39153)
I20260809 16:21:14.781177 21710 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrNqUCn/build/release/bin/kudu as pid 4637
I20260809 16:21:14.781270 21710 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786292417722232-21710-0/minicluster-data/ts-0/wal/instance
I20260809 16:21:14.781515 21710 external_mini_cluster.cc:1428] Setting key 26a07d4d0ab6280bf04b2c1a4122e088
I20260809 16:21:14.782428 21710 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrNqUCn/build/release/bin/kudu
/tmp/dist-test-taskrNqUCn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786292417722232-21710-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786292417722232-21710-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786292417722232-21710-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786292417722232-21710-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.51.130:0
--local_ip_for_outbound_sockets=127.21.51.130
--webserver_interface=127.21.51.130
--webserver_port=0
--tserver_master_addrs=127.21.51.190:45221
--builtin_ntp_servers=127.21.51.148:41717
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260809 16:21:14.864735 4823 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:21:14.864934 4823 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:21:14.864955 4823 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:21:14.866045 4823 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:21:14.866082 4823 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:21:14.866106 4823 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.51.130
I20260809 16:21:14.867384 4823 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.51.148:41717
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786292417722232-21710-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786292417722232-21710-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.51.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786292417722232-21710-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.51.130
--webserver_port=0
--tserver_master_addrs=127.21.51.190:45221
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4823
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.51.130
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786292417722232-21710-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:21:14.867569 4823 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:21:14.867816 4823 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:21:14.873047 4831 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:21:14.873091 4823 server_base.cc:1034] running on GCE node
W20260809 16:21:14.873013 4833 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:21:14.873050 4829 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:21:14.873415 4823 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:21:14.873656 4823 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:21:14.874816 4823 hybrid_clock.cc:648] HybridClock initialized: now 1786292474874803 us; error 29 us; skew 500 ppm
I20260809 16:21:14.876019 4823 webserver.cc:460] Webserver started at http://127.21.51.130:43657/ using document root <none> and password file <none>
I20260809 16:21:14.876268 4823 fs_manager.cc:359] Metadata directory not provided
I20260809 16:21:14.876317 4823 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:21:14.876487 4823 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:21:14.877631 4823 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786292417722232-21710-0/minicluster-data/ts-1/data/instance:
uuid: "759efa5c2ec14c6581740bd496baddc3"
format_stamp: "Formatted at 2026-08-09 16:21:14 on dist-test-slave-s11t"
server_key: "52ccbd3e4452d93af53c873021f234e7"
server_key_iv: "bf43007701ee56d4f830d45256b3712d"
server_key_version: "encryptionkey@0"
I20260809 16:21:14.878021 4823 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786292417722232-21710-0/minicluster-data/ts-1/wal/instance:
uuid: "759efa5c2ec14c6581740bd496baddc3"
format_stamp: "Formatted at 2026-08-09 16:21:14 on dist-test-slave-s11t"
server_key: "52ccbd3e4452d93af53c873021f234e7"
server_key_iv: "bf43007701ee56d4f830d45256b3712d"
server_key_version: "encryptionkey@0"
I20260809 16:21:14.879277 4823 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 16:21:14.880002 4842 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:21:14.880167 4823 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 16:21:14.880283 4823 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786292417722232-21710-0/minicluster-data/ts-1/data,/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786292417722232-21710-0/minicluster-data/ts-1/wal
uuid: "759efa5c2ec14c6581740bd496baddc3"
format_stamp: "Formatted at 2026-08-09 16:21:14 on dist-test-slave-s11t"
server_key: "52ccbd3e4452d93af53c873021f234e7"
server_key_iv: "bf43007701ee56d4f830d45256b3712d"
server_key_version: "encryptionkey@0"
I20260809 16:21:14.880373 4823 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786292417722232-21710-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786292417722232-21710-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786292417722232-21710-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:21:14.883742 4823 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:21:14.883997 4823 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:21:14.884142 4823 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:21:14.884354 4823 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:21:14.884654 4823 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:21:14.884728 4823 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:21:14.884768 4823 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:21:14.884797 4823 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:21:14.890558 4823 rpc_server.cc:307] RPC server started. Bound to: 127.21.51.130:40897
I20260809 16:21:14.890632 5033 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.51.130:40897 every 8 connection(s)
I20260809 16:21:14.890935 4823 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786292417722232-21710-0/minicluster-data/ts-1/data/info.pb
I20260809 16:21:14.893977 4609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:21:14.891640 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.51.130:53225 (local address 127.21.51.190:45221)
0809 16:21:14.891689 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0809 16:21:14.891691 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:21:14.891939 (+ 248us) server_negotiation.cc:407] Connection header received
0809 16:21:14.892425 (+ 486us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:21:14.892427 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:21:14.892434 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:21:14.892497 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:21:14.893714 (+ 1217us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:21:14.893716 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:21:14.893716 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 16:21:14.893721 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:21:14.893739 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 16:21:14.893744 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:21:14.893810 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0809 16:21:14.893890 (+ 80us) server_negotiation.cc:299] Negotiation successful
0809 16:21:14.893905 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260809 16:21:14.894171 5041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:21:14.891630 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:45221 (local address 127.21.51.130:53225)
0809 16:21:14.891825 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0809 16:21:14.891855 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0809 16:21:14.892302 (+ 447us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:21:14.892570 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:21:14.892574 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:21:14.892586 (+ 12us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:21:14.893557 (+ 971us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 16:21:14.893557 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:21:14.893584 (+ 27us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 16:21:14.893611 (+ 27us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:21:14.893820 (+ 209us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:21:14.893826 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:21:14.893829 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 16:21:14.893901 (+ 72us) client_negotiation.cc:240] Negotiation successful
0809 16:21:14.893960 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":92,"threads_started":1}
I20260809 16:21:14.894649 5035 heartbeater.cc:347] Connected to a master server at 127.21.51.190:45221
I20260809 16:21:14.894721 5035 heartbeater.cc:464] Registering TS with master...
I20260809 16:21:14.894873 5035 heartbeater.cc:511] Master 127.21.51.190:45221 requested a full tablet report, sending...
I20260809 16:21:14.895226 4540 ts_manager.cc:194] Registered new tserver with Master: 759efa5c2ec14c6581740bd496baddc3 (127.21.51.130:40897)
I20260809 16:21:14.897737 21710 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrNqUCn/build/release/bin/kudu as pid 4823
I20260809 16:21:14.897819 21710 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786292417722232-21710-0/minicluster-data/ts-1/wal/instance
I20260809 16:21:14.897953 21710 external_mini_cluster.cc:1428] Setting key 78e697146e78f310df16ad1a0bd81ecd
I20260809 16:21:14.898697 21710 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrNqUCn/build/release/bin/kudu
/tmp/dist-test-taskrNqUCn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786292417722232-21710-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786292417722232-21710-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786292417722232-21710-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786292417722232-21710-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.51.131:0
--local_ip_for_outbound_sockets=127.21.51.131
--webserver_interface=127.21.51.131
--webserver_port=0
--tserver_master_addrs=127.21.51.190:45221
--builtin_ntp_servers=127.21.51.148:41717
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260809 16:21:14.980082 5043 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:21:14.980284 5043 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:21:14.980306 5043 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:21:14.981492 5043 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:21:14.981535 5043 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:21:14.981561 5043 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.51.131
I20260809 16:21:14.982983 5043 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.51.148:41717
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786292417722232-21710-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786292417722232-21710-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.51.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786292417722232-21710-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.51.131
--webserver_port=0
--tserver_master_addrs=127.21.51.190:45221
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5043
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.51.131
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786292417722232-21710-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:21:14.983227 5043 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:21:14.983547 5043 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:21:14.988718 5052 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:21:14.988718 5056 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:21:14.988781 5053 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:21:14.989111 5043 server_base.cc:1034] running on GCE node
I20260809 16:21:14.989295 5043 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:21:14.989487 5043 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:21:14.990622 5043 hybrid_clock.cc:648] HybridClock initialized: now 1786292474990612 us; error 31 us; skew 500 ppm
I20260809 16:21:14.991892 5043 webserver.cc:460] Webserver started at http://127.21.51.131:43083/ using document root <none> and password file <none>
I20260809 16:21:14.992084 5043 fs_manager.cc:359] Metadata directory not provided
I20260809 16:21:14.992125 5043 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:21:14.992218 5043 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:21:14.993285 5043 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786292417722232-21710-0/minicluster-data/ts-2/data/instance:
uuid: "e5ee4d75f65a4d5d8e69d91b3531c5ed"
format_stamp: "Formatted at 2026-08-09 16:21:14 on dist-test-slave-s11t"
server_key: "6e0fba9f389ccf35c4aa273e45720eb6"
server_key_iv: "20eced513f8e591e7d7b10094ae1c856"
server_key_version: "encryptionkey@0"
I20260809 16:21:14.993678 5043 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786292417722232-21710-0/minicluster-data/ts-2/wal/instance:
uuid: "e5ee4d75f65a4d5d8e69d91b3531c5ed"
format_stamp: "Formatted at 2026-08-09 16:21:14 on dist-test-slave-s11t"
server_key: "6e0fba9f389ccf35c4aa273e45720eb6"
server_key_iv: "20eced513f8e591e7d7b10094ae1c856"
server_key_version: "encryptionkey@0"
I20260809 16:21:14.994985 5043 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 16:21:14.995689 5071 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:21:14.995875 5043 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 16:21:14.995962 5043 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786292417722232-21710-0/minicluster-data/ts-2/data,/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786292417722232-21710-0/minicluster-data/ts-2/wal
uuid: "e5ee4d75f65a4d5d8e69d91b3531c5ed"
format_stamp: "Formatted at 2026-08-09 16:21:14 on dist-test-slave-s11t"
server_key: "6e0fba9f389ccf35c4aa273e45720eb6"
server_key_iv: "20eced513f8e591e7d7b10094ae1c856"
server_key_version: "encryptionkey@0"
I20260809 16:21:14.996016 5043 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786292417722232-21710-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786292417722232-21710-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786292417722232-21710-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:21:14.999274 5043 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:21:14.999471 5043 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:21:14.999562 5043 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:21:14.999716 5043 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:21:14.999969 5043 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:21:15.000000 5043 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:21:15.000020 5043 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:21:15.000033 5043 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:21:15.005607 5043 rpc_server.cc:307] RPC server started. Bound to: 127.21.51.131:33351
I20260809 16:21:15.005662 5233 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.51.131:33351 every 8 connection(s)
I20260809 16:21:15.005995 5043 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786292417722232-21710-0/minicluster-data/ts-2/data/info.pb
I20260809 16:21:15.009327 4609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:21:15.006737 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.51.131:53617 (local address 127.21.51.190:45221)
0809 16:21:15.006798 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0809 16:21:15.006800 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:21:15.007068 (+ 268us) server_negotiation.cc:407] Connection header received
0809 16:21:15.007622 (+ 554us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:21:15.007624 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:21:15.007631 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:21:15.007682 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:21:15.009032 (+ 1350us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:21:15.009034 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:21:15.009034 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 16:21:15.009039 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:21:15.009055 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 16:21:15.009060 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:21:15.009139 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0809 16:21:15.009234 (+ 95us) server_negotiation.cc:299] Negotiation successful
0809 16:21:15.009253 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260809 16:21:15.009533 5237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:21:15.006721 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:45221 (local address 127.21.51.131:53617)
0809 16:21:15.006924 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0809 16:21:15.006956 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0809 16:21:15.007504 (+ 548us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:21:15.007752 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:21:15.007761 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:21:15.007768 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:21:15.008875 (+ 1107us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 16:21:15.008875 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:21:15.008906 (+ 31us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 16:21:15.008936 (+ 30us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:21:15.009135 (+ 199us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:21:15.009141 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:21:15.009144 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 16:21:15.009244 (+ 100us) client_negotiation.cc:240] Negotiation successful
0809 16:21:15.009308 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":91,"threads_started":1}
I20260809 16:21:15.010192 5235 heartbeater.cc:347] Connected to a master server at 127.21.51.190:45221
I20260809 16:21:15.010309 5235 heartbeater.cc:464] Registering TS with master...
I20260809 16:21:15.010469 5235 heartbeater.cc:511] Master 127.21.51.190:45221 requested a full tablet report, sending...
I20260809 16:21:15.010915 4540 ts_manager.cc:194] Registered new tserver with Master: e5ee4d75f65a4d5d8e69d91b3531c5ed (127.21.51.131:33351)
I20260809 16:21:15.014000 21710 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrNqUCn/build/release/bin/kudu as pid 5043
I20260809 16:21:15.014079 21710 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786292417722232-21710-0/minicluster-data/ts-2/wal/instance
I20260809 16:21:15.014250 21710 external_mini_cluster.cc:1428] Setting key 442590b512b6e51fee800d146f58249c
I20260809 16:21:15.015180 21710 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260809 16:21:15.017064 4609 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:21:15.016500 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52284 (local address 127.21.51.190:45221)
0809 16:21:15.016579 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0809 16:21:15.016581 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:21:15.016673 (+ 92us) server_negotiation.cc:407] Connection header received
0809 16:21:15.016810 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:21:15.016811 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:21:15.016815 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:21:15.016869 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:21:15.016981 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52284: BlockingRecv error: recv got EOF from 127.0.0.1:52284 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
I20260809 16:21:15.017175 21710 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrNqUCn/build/release/bin/kudu with pid 4637
I20260809 16:21:15.039687 21710 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrNqUCn/build/release/bin/kudu with pid 4823
I20260809 16:21:15.047197 21710 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrNqUCn/build/release/bin/kudu with pid 5043
I20260809 16:21:15.057263 21710 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrNqUCn/build/release/bin/kudu with pid 4485
2026-08-09T16:21:15Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (583 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786292417722232-21710-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 16:21:15 dist-test-slave-s11t krb5kdc[5249](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 16:21:15 dist-test-slave-s11t krb5kdc[5249](info): set up 2 sockets
Aug 09 16:21:15 dist-test-slave-s11t krb5kdc[5249](info): commencing operation
krb5kdc: starting...
W20260809 16:21:16.296146 21710 mini_kdc.cc:121] Time spent starting KDC: real 1.215s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786292417722232-21710-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786292417722232-21710-0/krb5kdc/test-user.keytab.
Aug 09 16:21:16 dist-test-slave-s11t krb5kdc[5249](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292476, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T16:21:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T16:21:16Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.51.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.51.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.51.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.190.keytab.
Entry for principal kudu/127.21.51.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.190.keytab.
I20260809 16:21:16.417315 21710 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrNqUCn/build/release/bin/kudu
/tmp/dist-test-taskrNqUCn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786292417722232-21710-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786292417722232-21710-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786292417722232-21710-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786292417722232-21710-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.51.190:41021
--webserver_interface=127.21.51.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.190.keytab
--principal=kudu/127.21.51.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.51.148:36085
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.51.190:41021
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786292417722232-21710-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786292417722232-21710-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786292417722232-21710-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:21:16.499409 5265 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:21:16.499646 5265 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:21:16.499682 5265 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:21:16.500835 5265 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 16:21:16.500907 5265 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:21:16.500936 5265 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:21:16.500967 5265 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 16:21:16.500991 5265 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 16:21:16.502452 5265 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.51.148:36085
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786292417722232-21710-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786292417722232-21710-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.51.190:41021
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.190.keytab
--principal=kudu/127.21.51.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.51.190:41021
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786292417722232-21710-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.51.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5265
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786292417722232-21710-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:21:16.502736 5265 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:21:16.502970 5265 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:21:16.507858 5275 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:21:16.507869 5282 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:21:16.507902 5273 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:21:16.508038 5265 server_base.cc:1034] running on GCE node
I20260809 16:21:16.508258 5265 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:21:16.508558 5265 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:21:16.509702 5265 hybrid_clock.cc:648] HybridClock initialized: now 1786292476509680 us; error 41 us; skew 500 ppm
Aug 09 16:21:16 dist-test-slave-s11t krb5kdc[5249](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292476, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.51.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:21:16.512355 5265 init.cc:377] Logged in from keytab as kudu/127.21.51.190@KRBTEST.COM (short username kudu)
I20260809 16:21:16.512892 5265 webserver.cc:460] Webserver started at http://127.21.51.190:37347/ using document root <none> and password file <none>
I20260809 16:21:16.513124 5265 fs_manager.cc:359] Metadata directory not provided
I20260809 16:21:16.513191 5265 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:21:16.513356 5265 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:21:16.514418 5265 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786292417722232-21710-0/minicluster-data/master-0/data/instance:
uuid: "4a24add1b1cc4881ae9579132b4c8dec"
format_stamp: "Formatted at 2026-08-09 16:21:16 on dist-test-slave-s11t"
server_key: "5481673fbf3498e24e7a74f2fd7ee964"
server_key_iv: "750aab139272836a8ab98279b8f0740e"
server_key_version: "encryptionkey@0"
I20260809 16:21:16.514726 5265 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786292417722232-21710-0/minicluster-data/master-0/wal/instance:
uuid: "4a24add1b1cc4881ae9579132b4c8dec"
format_stamp: "Formatted at 2026-08-09 16:21:16 on dist-test-slave-s11t"
server_key: "5481673fbf3498e24e7a74f2fd7ee964"
server_key_iv: "750aab139272836a8ab98279b8f0740e"
server_key_version: "encryptionkey@0"
I20260809 16:21:16.516016 5265 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 16:21:16.516672 5291 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:21:16.516939 5265 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:21:16.517042 5265 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786292417722232-21710-0/minicluster-data/master-0/data,/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786292417722232-21710-0/minicluster-data/master-0/wal
uuid: "4a24add1b1cc4881ae9579132b4c8dec"
format_stamp: "Formatted at 2026-08-09 16:21:16 on dist-test-slave-s11t"
server_key: "5481673fbf3498e24e7a74f2fd7ee964"
server_key_iv: "750aab139272836a8ab98279b8f0740e"
server_key_version: "encryptionkey@0"
I20260809 16:21:16.517133 5265 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786292417722232-21710-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786292417722232-21710-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786292417722232-21710-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:21:16.527309 5265 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:21:16.529613 5265 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:21:16.529786 5265 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:21:16.534210 5265 rpc_server.cc:307] RPC server started. Bound to: 127.21.51.190:41021
I20260809 16:21:16.534251 5361 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.51.190:41021 every 8 connection(s)
I20260809 16:21:16.534624 5265 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786292417722232-21710-0/minicluster-data/master-0/data/info.pb
I20260809 16:21:16.535754 5363 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:21:16.539870 5363 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4a24add1b1cc4881ae9579132b4c8dec: Bootstrap starting.
I20260809 16:21:16.540735 5363 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4a24add1b1cc4881ae9579132b4c8dec: Neither blocks nor log segments found. Creating new log.
I20260809 16:21:16.541105 5363 log.cc:824] T 00000000000000000000000000000000 P 4a24add1b1cc4881ae9579132b4c8dec: Log is configured to *not* fsync() on all Append() calls
I20260809 16:21:16.542019 21710 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrNqUCn/build/release/bin/kudu as pid 5265
I20260809 16:21:16.542124 21710 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786292417722232-21710-0/minicluster-data/master-0/wal/instance
I20260809 16:21:16.542220 5363 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4a24add1b1cc4881ae9579132b4c8dec: No bootstrap required, opened a new log
I20260809 16:21:16.542328 21710 external_mini_cluster.cc:1428] Setting key 7eab4d15951eb2c864505ed8d754c34e
I20260809 16:21:16.545176 5363 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4a24add1b1cc4881ae9579132b4c8dec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a24add1b1cc4881ae9579132b4c8dec" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 41021 } }
I20260809 16:21:16.545395 5363 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4a24add1b1cc4881ae9579132b4c8dec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:21:16.545450 5363 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4a24add1b1cc4881ae9579132b4c8dec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4a24add1b1cc4881ae9579132b4c8dec, State: Initialized, Role: FOLLOWER
I20260809 16:21:16.545590 5363 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4a24add1b1cc4881ae9579132b4c8dec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a24add1b1cc4881ae9579132b4c8dec" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 41021 } }
I20260809 16:21:16.545699 5363 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4a24add1b1cc4881ae9579132b4c8dec [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 16:21:16.545749 5363 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4a24add1b1cc4881ae9579132b4c8dec [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 16:21:16.545797 5363 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4a24add1b1cc4881ae9579132b4c8dec [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:21:16.546743 5363 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4a24add1b1cc4881ae9579132b4c8dec [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a24add1b1cc4881ae9579132b4c8dec" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 41021 } }
I20260809 16:21:16.546880 5363 leader_election.cc:302] T 00000000000000000000000000000000 P 4a24add1b1cc4881ae9579132b4c8dec [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4a24add1b1cc4881ae9579132b4c8dec; no voters:
I20260809 16:21:16.547046 5363 leader_election.cc:288] T 00000000000000000000000000000000 P 4a24add1b1cc4881ae9579132b4c8dec [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 16:21:16.547071 5375 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4a24add1b1cc4881ae9579132b4c8dec [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:21:16.547204 5375 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4a24add1b1cc4881ae9579132b4c8dec [term 1 LEADER]: Becoming Leader. State: Replica: 4a24add1b1cc4881ae9579132b4c8dec, State: Running, Role: LEADER
I20260809 16:21:16.547318 5363 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4a24add1b1cc4881ae9579132b4c8dec [sys.catalog]: configured and running, proceeding with master startup.
I20260809 16:21:16.547305 5375 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4a24add1b1cc4881ae9579132b4c8dec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a24add1b1cc4881ae9579132b4c8dec" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 41021 } }
I20260809 16:21:16.547669 5375 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4a24add1b1cc4881ae9579132b4c8dec [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4a24add1b1cc4881ae9579132b4c8dec. Latest consensus state: current_term: 1 leader_uuid: "4a24add1b1cc4881ae9579132b4c8dec" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a24add1b1cc4881ae9579132b4c8dec" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 41021 } } }
I20260809 16:21:16.547674 5377 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4a24add1b1cc4881ae9579132b4c8dec [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4a24add1b1cc4881ae9579132b4c8dec" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a24add1b1cc4881ae9579132b4c8dec" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 41021 } } }
I20260809 16:21:16.547761 5375 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4a24add1b1cc4881ae9579132b4c8dec [sys.catalog]: This master's current role is: LEADER
I20260809 16:21:16.547787 5377 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4a24add1b1cc4881ae9579132b4c8dec [sys.catalog]: This master's current role is: LEADER
W20260809 16:21:16.549530 5404 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 4a24add1b1cc4881ae9579132b4c8dec: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 16:21:16.549727 5404 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 16:21:16.549822 5381 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 16:21:16.550129 5381 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 09 16:21:16 dist-test-slave-s11t krb5kdc[5249](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292476, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.51.190@KRBTEST.COM
I20260809 16:21:16.552594 5381 catalog_manager.cc:1347] Generated new cluster ID: ffa1d9da72174dbe85559824d5f952ae
I20260809 16:21:16.552646 5381 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 16:21:16.557700 5369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:21:16.543058 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42046 (local address 127.21.51.190:41021)
0809 16:21:16.545108 (+ 2050us) server_negotiation.cc:206] Beginning negotiation
0809 16:21:16.545120 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 16:21:16.545166 (+ 46us) server_negotiation.cc:407] Connection header received
0809 16:21:16.545315 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:21:16.545326 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:21:16.545404 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:21:16.545595 (+ 191us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:21:16.546399 (+ 804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:21:16.548253 (+ 1854us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:21:16.549551 (+ 1298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:21:16.550225 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:21:16.550334 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:21:16.553115 (+ 2781us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:21:16.553119 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:21:16.553119 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:21:16.553133 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:21:16.555300 (+ 2167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:21:16.555757 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:21:16.555760 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:21:16.555760 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:21:16.555905 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:21:16.556138 (+ 233us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:21:16.556139 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:21:16.556139 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:21:16.556484 (+ 345us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:21:16.556651 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0809 16:21:16.557220 (+ 569us) server_negotiation.cc:299] Negotiation successful
0809 16:21:16.557417 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1982,"thread_start_us":1944,"threads_started":1}
I20260809 16:21:16.564834 5381 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 16:21:16.565444 5381 catalog_manager.cc:1504] Loading token signing keys...
I20260809 16:21:16.585417 5381 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4a24add1b1cc4881ae9579132b4c8dec: Generated new TSK 0
I20260809 16:21:16.585621 5381 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.21.51.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.51.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.51.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.129.keytab.
Entry for principal kudu/127.21.51.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.129.keytab.
I20260809 16:21:16.620016 21710 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrNqUCn/build/release/bin/kudu
/tmp/dist-test-taskrNqUCn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786292417722232-21710-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786292417722232-21710-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786292417722232-21710-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786292417722232-21710-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.51.129:0
--local_ip_for_outbound_sockets=127.21.51.129
--webserver_interface=127.21.51.129
--webserver_port=0
--tserver_master_addrs=127.21.51.190:41021
--keytab_file=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.129.keytab
--principal=kudu/127.21.51.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.51.148:36085
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786292417722232-21710-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786292417722232-21710-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786292417722232-21710-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:21:16.703598 5411 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:21:16.703796 5411 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:21:16.703819 5411 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:21:16.705015 5411 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:21:16.705057 5411 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:21:16.705084 5411 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.51.129
I20260809 16:21:16.706504 5411 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.51.148:36085
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786292417722232-21710-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786292417722232-21710-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.129.keytab
--principal=kudu/127.21.51.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.51.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786292417722232-21710-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.51.129
--webserver_port=0
--tserver_master_addrs=127.21.51.190:41021
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5411
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.51.129
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786292417722232-21710-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:21:16.706722 5411 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:21:16.706947 5411 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:21:16.712034 5419 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:21:16.712019 5421 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:21:16.712090 5411 server_base.cc:1034] running on GCE node
W20260809 16:21:16.712019 5418 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:21:16.712456 5411 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:21:16.712724 5411 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:21:16.713886 5411 hybrid_clock.cc:648] HybridClock initialized: now 1786292476713867 us; error 35 us; skew 500 ppm
Aug 09 16:21:16 dist-test-slave-s11t krb5kdc[5249](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292476, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.51.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:21:16.716415 5411 init.cc:377] Logged in from keytab as kudu/127.21.51.129@KRBTEST.COM (short username kudu)
I20260809 16:21:16.716921 5411 webserver.cc:460] Webserver started at http://127.21.51.129:45331/ using document root <none> and password file <none>
I20260809 16:21:16.717144 5411 fs_manager.cc:359] Metadata directory not provided
I20260809 16:21:16.717190 5411 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:21:16.717346 5411 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:21:16.718438 5411 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786292417722232-21710-0/minicluster-data/ts-0/data/instance:
uuid: "df77827a888a44caab9d71371a152972"
format_stamp: "Formatted at 2026-08-09 16:21:16 on dist-test-slave-s11t"
server_key: "a6e710936be0c5a0902d7a7727f8e5d1"
server_key_iv: "0857419b85dcc67832081bf96834528f"
server_key_version: "encryptionkey@0"
I20260809 16:21:16.718812 5411 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786292417722232-21710-0/minicluster-data/ts-0/wal/instance:
uuid: "df77827a888a44caab9d71371a152972"
format_stamp: "Formatted at 2026-08-09 16:21:16 on dist-test-slave-s11t"
server_key: "a6e710936be0c5a0902d7a7727f8e5d1"
server_key_iv: "0857419b85dcc67832081bf96834528f"
server_key_version: "encryptionkey@0"
I20260809 16:21:16.720302 5411 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 16:21:16.721199 5433 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:21:16.721413 5411 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260809 16:21:16.721515 5411 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786292417722232-21710-0/minicluster-data/ts-0/data,/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786292417722232-21710-0/minicluster-data/ts-0/wal
uuid: "df77827a888a44caab9d71371a152972"
format_stamp: "Formatted at 2026-08-09 16:21:16 on dist-test-slave-s11t"
server_key: "a6e710936be0c5a0902d7a7727f8e5d1"
server_key_iv: "0857419b85dcc67832081bf96834528f"
server_key_version: "encryptionkey@0"
I20260809 16:21:16.721599 5411 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786292417722232-21710-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786292417722232-21710-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786292417722232-21710-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:21:16.735544 5411 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:21:16.737983 5411 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:21:16.738157 5411 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:21:16.738409 5411 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:21:16.738744 5411 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:21:16.738801 5411 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:21:16.738845 5411 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:21:16.738873 5411 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:21:16.745005 5411 rpc_server.cc:307] RPC server started. Bound to: 127.21.51.129:37605
I20260809 16:21:16.745050 5612 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.51.129:37605 every 8 connection(s)
I20260809 16:21:16.745374 5411 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786292417722232-21710-0/minicluster-data/ts-0/data/info.pb
I20260809 16:21:16.746027 21710 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrNqUCn/build/release/bin/kudu as pid 5411
I20260809 16:21:16.746145 21710 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786292417722232-21710-0/minicluster-data/ts-0/wal/instance
I20260809 16:21:16.746408 21710 external_mini_cluster.cc:1428] Setting key 8ccd3ab941caef8aba07505d0dd2cffb
Aug 09 16:21:16 dist-test-slave-s11t krb5kdc[5249](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292476, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.51.129@KRBTEST.COM for kudu/127.21.51.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.21.51.130@KRBTEST.COM; defaulting to no policy
I20260809 16:21:16.756889 5369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:21:16.746168 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.51.129:47697 (local address 127.21.51.190:41021)
0809 16:21:16.746478 (+ 310us) server_negotiation.cc:206] Beginning negotiation
0809 16:21:16.746481 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 16:21:16.746497 (+ 16us) server_negotiation.cc:407] Connection header received
0809 16:21:16.747077 (+ 580us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:21:16.747079 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:21:16.747094 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:21:16.747211 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:21:16.748650 (+ 1439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:21:16.749593 (+ 943us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:21:16.750613 (+ 1020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:21:16.750967 (+ 354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:21:16.751048 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:21:16.753769 (+ 2721us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:21:16.753770 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:21:16.753771 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:21:16.753779 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:21:16.755436 (+ 1657us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:21:16.755820 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:21:16.755822 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:21:16.755822 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:21:16.755888 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:21:16.756158 (+ 270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:21:16.756159 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:21:16.756159 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:21:16.756310 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:21:16.756390 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0809 16:21:16.756731 (+ 341us) server_negotiation.cc:299] Negotiation successful
0809 16:21:16.756821 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":273}
I20260809 16:21:16.757076 5617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:21:16.746151 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:41021 (local address 127.21.51.129:47697)
0809 16:21:16.746349 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0809 16:21:16.746386 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0809 16:21:16.746970 (+ 584us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:21:16.747280 (+ 310us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:21:16.747285 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:21:16.748236 (+ 951us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:21:16.748566 (+ 330us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:21:16.748572 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:21:16.749688 (+ 1116us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:21:16.749689 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:21:16.750513 (+ 824us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:21:16.750515 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:21:16.751085 (+ 570us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:21:16.751086 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:21:16.751389 (+ 303us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:21:16.751969 (+ 580us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:21:16.751978 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:21:16.753664 (+ 1686us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:21:16.755517 (+ 1853us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:21:16.755523 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:21:16.755525 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:21:16.755737 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:21:16.755963 (+ 226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:21:16.755964 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:21:16.755964 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:21:16.756059 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:21:16.756410 (+ 351us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:21:16.756418 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:21:16.756566 (+ 148us) client_negotiation.cc:769] Sending connection context
0809 16:21:16.756654 (+ 88us) client_negotiation.cc:240] Negotiation successful
0809 16:21:16.756843 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":126,"threads_started":1}
I20260809 16:21:16.757647 5613 heartbeater.cc:347] Connected to a master server at 127.21.51.190:41021
I20260809 16:21:16.757743 5613 heartbeater.cc:464] Registering TS with master...
I20260809 16:21:16.757889 5613 heartbeater.cc:511] Master 127.21.51.190:41021 requested a full tablet report, sending...
I20260809 16:21:16.758405 5314 ts_manager.cc:194] Registered new tserver with Master: df77827a888a44caab9d71371a152972 (127.21.51.129:37605)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.51.130@KRBTEST.COM" created.
I20260809 16:21:16.759045 5314 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.51.129@KRBTEST.COM'} at 127.21.51.129:47697
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.51.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.130.keytab.
Entry for principal kudu/127.21.51.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.130.keytab.
I20260809 16:21:16.774050 21710 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrNqUCn/build/release/bin/kudu
/tmp/dist-test-taskrNqUCn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786292417722232-21710-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786292417722232-21710-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786292417722232-21710-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786292417722232-21710-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.51.130:0
--local_ip_for_outbound_sockets=127.21.51.130
--webserver_interface=127.21.51.130
--webserver_port=0
--tserver_master_addrs=127.21.51.190:41021
--keytab_file=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.130.keytab
--principal=kudu/127.21.51.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.51.148:36085
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786292417722232-21710-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786292417722232-21710-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786292417722232-21710-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:21:16.856942 5621 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:21:16.857138 5621 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:21:16.857161 5621 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:21:16.858218 5621 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:21:16.858263 5621 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:21:16.858289 5621 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.51.130
I20260809 16:21:16.859733 5621 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.51.148:36085
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786292417722232-21710-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786292417722232-21710-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.130.keytab
--principal=kudu/127.21.51.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.51.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786292417722232-21710-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.51.130
--webserver_port=0
--tserver_master_addrs=127.21.51.190:41021
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5621
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.51.130
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786292417722232-21710-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:21:16.859917 5621 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:21:16.860097 5621 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:21:16.865108 5629 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:21:16.865113 5631 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:21:16.865113 5627 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:21:16.865350 5621 server_base.cc:1034] running on GCE node
I20260809 16:21:16.865550 5621 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:21:16.865733 5621 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:21:16.866901 5621 hybrid_clock.cc:648] HybridClock initialized: now 1786292476866894 us; error 33 us; skew 500 ppm
Aug 09 16:21:16 dist-test-slave-s11t krb5kdc[5249](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292476, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.51.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:21:16.869463 5621 init.cc:377] Logged in from keytab as kudu/127.21.51.130@KRBTEST.COM (short username kudu)
I20260809 16:21:16.869882 5621 webserver.cc:460] Webserver started at http://127.21.51.130:33933/ using document root <none> and password file <none>
I20260809 16:21:16.870108 5621 fs_manager.cc:359] Metadata directory not provided
I20260809 16:21:16.870177 5621 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:21:16.870330 5621 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:21:16.871337 5621 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786292417722232-21710-0/minicluster-data/ts-1/data/instance:
uuid: "63fb26bc10de499389d505ec1d22f22c"
format_stamp: "Formatted at 2026-08-09 16:21:16 on dist-test-slave-s11t"
server_key: "c9fe42ea851fe9978a9cac7df284cd94"
server_key_iv: "13704fc2940a7b838ad20dfed37c136a"
server_key_version: "encryptionkey@0"
I20260809 16:21:16.871709 5621 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786292417722232-21710-0/minicluster-data/ts-1/wal/instance:
uuid: "63fb26bc10de499389d505ec1d22f22c"
format_stamp: "Formatted at 2026-08-09 16:21:16 on dist-test-slave-s11t"
server_key: "c9fe42ea851fe9978a9cac7df284cd94"
server_key_iv: "13704fc2940a7b838ad20dfed37c136a"
server_key_version: "encryptionkey@0"
I20260809 16:21:16.873131 5621 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260809 16:21:16.873868 5648 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:21:16.874037 5621 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 16:21:16.874146 5621 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786292417722232-21710-0/minicluster-data/ts-1/data,/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786292417722232-21710-0/minicluster-data/ts-1/wal
uuid: "63fb26bc10de499389d505ec1d22f22c"
format_stamp: "Formatted at 2026-08-09 16:21:16 on dist-test-slave-s11t"
server_key: "c9fe42ea851fe9978a9cac7df284cd94"
server_key_iv: "13704fc2940a7b838ad20dfed37c136a"
server_key_version: "encryptionkey@0"
I20260809 16:21:16.874221 5621 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786292417722232-21710-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786292417722232-21710-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786292417722232-21710-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:21:16.884326 5621 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:21:16.886173 5621 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:21:16.886339 5621 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:21:16.886572 5621 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:21:16.886878 5621 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:21:16.886933 5621 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:21:16.886967 5621 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:21:16.886996 5621 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:21:16.892619 5621 rpc_server.cc:307] RPC server started. Bound to: 127.21.51.130:36569
I20260809 16:21:16.892729 5829 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.51.130:36569 every 8 connection(s)
I20260809 16:21:16.893044 5621 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786292417722232-21710-0/minicluster-data/ts-1/data/info.pb
Aug 09 16:21:16 dist-test-slave-s11t krb5kdc[5249](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292476, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.51.130@KRBTEST.COM for kudu/127.21.51.190@KRBTEST.COM
I20260809 16:21:16.899008 21710 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrNqUCn/build/release/bin/kudu as pid 5621
I20260809 16:21:16.899113 21710 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786292417722232-21710-0/minicluster-data/ts-1/wal/instance
I20260809 16:21:16.899356 21710 external_mini_cluster.cc:1428] Setting key e3d468c0af35c3bda0b68657d8aee7be
I20260809 16:21:16.902307 5369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:21:16.893776 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.51.130:52991 (local address 127.21.51.190:41021)
0809 16:21:16.893824 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0809 16:21:16.893826 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:21:16.894031 (+ 205us) server_negotiation.cc:407] Connection header received
0809 16:21:16.894495 (+ 464us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:21:16.894496 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:21:16.894505 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:21:16.894564 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:21:16.895454 (+ 890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:21:16.895948 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:21:16.896913 (+ 965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:21:16.897102 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:21:16.897160 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:21:16.899306 (+ 2146us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:21:16.899308 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:21:16.899308 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:21:16.899314 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:21:16.900831 (+ 1517us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:21:16.901348 (+ 517us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:21:16.901348 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:21:16.901349 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:21:16.901402 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:21:16.901683 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:21:16.901684 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:21:16.901684 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:21:16.901783 (+ 99us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:21:16.901836 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0809 16:21:16.902178 (+ 342us) server_negotiation.cc:299] Negotiation successful
0809 16:21:16.902245 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260809 16:21:16.902462 5832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:21:16.893735 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:41021 (local address 127.21.51.130:52991)
0809 16:21:16.893905 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0809 16:21:16.893945 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0809 16:21:16.894412 (+ 467us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:21:16.894634 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:21:16.894637 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:21:16.895190 (+ 553us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:21:16.895381 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:21:16.895385 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:21:16.896023 (+ 638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:21:16.896024 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:21:16.896835 (+ 811us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:21:16.896837 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:21:16.897178 (+ 341us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:21:16.897178 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:21:16.897274 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:21:16.897761 (+ 487us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:21:16.897774 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:21:16.899214 (+ 1440us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:21:16.900916 (+ 1702us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:21:16.900923 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:21:16.900926 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:21:16.901254 (+ 328us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:21:16.901470 (+ 216us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:21:16.901470 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:21:16.901471 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:21:16.901618 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:21:16.901851 (+ 233us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:21:16.901858 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:21:16.902039 (+ 181us) client_negotiation.cc:769] Sending connection context
0809 16:21:16.902152 (+ 113us) client_negotiation.cc:240] Negotiation successful
0809 16:21:16.902272 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":107,"threads_started":1}
I20260809 16:21:16.902974 5830 heartbeater.cc:347] Connected to a master server at 127.21.51.190:41021
I20260809 16:21:16.903070 5830 heartbeater.cc:464] Registering TS with master...
I20260809 16:21:16.903261 5830 heartbeater.cc:511] Master 127.21.51.190:41021 requested a full tablet report, sending...
I20260809 16:21:16.903651 5314 ts_manager.cc:194] Registered new tserver with Master: 63fb26bc10de499389d505ec1d22f22c (127.21.51.130:36569)
I20260809 16:21:16.904165 5314 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.51.130@KRBTEST.COM'} at 127.21.51.130:52991
WARNING: no policy specified for kudu/127.21.51.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.51.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.51.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.131.keytab.
Entry for principal kudu/127.21.51.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.131.keytab.
I20260809 16:21:16.926965 21710 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrNqUCn/build/release/bin/kudu
/tmp/dist-test-taskrNqUCn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786292417722232-21710-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786292417722232-21710-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786292417722232-21710-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786292417722232-21710-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.51.131:0
--local_ip_for_outbound_sockets=127.21.51.131
--webserver_interface=127.21.51.131
--webserver_port=0
--tserver_master_addrs=127.21.51.190:41021
--keytab_file=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.131.keytab
--principal=kudu/127.21.51.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.51.148:36085
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786292417722232-21710-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786292417722232-21710-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786292417722232-21710-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:21:17.009667 5835 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:21:17.009872 5835 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:21:17.009895 5835 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:21:17.011062 5835 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:21:17.011106 5835 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:21:17.011133 5835 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.51.131
I20260809 16:21:17.012600 5835 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.51.148:36085
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786292417722232-21710-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786292417722232-21710-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.131.keytab
--principal=kudu/127.21.51.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.51.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786292417722232-21710-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.51.131
--webserver_port=0
--tserver_master_addrs=127.21.51.190:41021
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5835
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.51.131
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786292417722232-21710-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:21:17.012888 5835 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:21:17.013152 5835 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:21:17.018311 5849 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:21:17.018354 5848 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:21:17.018517 5852 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:21:17.018493 5835 server_base.cc:1034] running on GCE node
I20260809 16:21:17.018736 5835 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:21:17.018990 5835 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:21:17.020211 5835 hybrid_clock.cc:648] HybridClock initialized: now 1786292477020197 us; error 31 us; skew 500 ppm
Aug 09 16:21:17 dist-test-slave-s11t krb5kdc[5249](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292477, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.51.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:21:17.022840 5835 init.cc:377] Logged in from keytab as kudu/127.21.51.131@KRBTEST.COM (short username kudu)
I20260809 16:21:17.023274 5835 webserver.cc:460] Webserver started at http://127.21.51.131:45453/ using document root <none> and password file <none>
I20260809 16:21:17.023538 5835 fs_manager.cc:359] Metadata directory not provided
I20260809 16:21:17.023586 5835 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:21:17.023741 5835 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:21:17.025002 5835 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786292417722232-21710-0/minicluster-data/ts-2/data/instance:
uuid: "a9be429d0ed24e5b86c8f333c1fbfb9f"
format_stamp: "Formatted at 2026-08-09 16:21:17 on dist-test-slave-s11t"
server_key: "33c72699bf14bc054a79da9bb3f1c958"
server_key_iv: "80a16f4562a6ad895c8efcba301c8d8a"
server_key_version: "encryptionkey@0"
I20260809 16:21:17.025372 5835 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786292417722232-21710-0/minicluster-data/ts-2/wal/instance:
uuid: "a9be429d0ed24e5b86c8f333c1fbfb9f"
format_stamp: "Formatted at 2026-08-09 16:21:17 on dist-test-slave-s11t"
server_key: "33c72699bf14bc054a79da9bb3f1c958"
server_key_iv: "80a16f4562a6ad895c8efcba301c8d8a"
server_key_version: "encryptionkey@0"
I20260809 16:21:17.026690 5835 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 16:21:17.027454 5862 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:21:17.027643 5835 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 16:21:17.027736 5835 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786292417722232-21710-0/minicluster-data/ts-2/data,/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786292417722232-21710-0/minicluster-data/ts-2/wal
uuid: "a9be429d0ed24e5b86c8f333c1fbfb9f"
format_stamp: "Formatted at 2026-08-09 16:21:17 on dist-test-slave-s11t"
server_key: "33c72699bf14bc054a79da9bb3f1c958"
server_key_iv: "80a16f4562a6ad895c8efcba301c8d8a"
server_key_version: "encryptionkey@0"
I20260809 16:21:17.027832 5835 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786292417722232-21710-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786292417722232-21710-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786292417722232-21710-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:21:17.048189 5835 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:21:17.050870 5835 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:21:17.051034 5835 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:21:17.051297 5835 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:21:17.051651 5835 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:21:17.051707 5835 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:21:17.051759 5835 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:21:17.051786 5835 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:21:17.057745 5835 rpc_server.cc:307] RPC server started. Bound to: 127.21.51.131:33095
I20260809 16:21:17.057804 6040 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.51.131:33095 every 8 connection(s)
I20260809 16:21:17.058171 5835 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786292417722232-21710-0/minicluster-data/ts-2/data/info.pb
I20260809 16:21:17.062263 21710 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrNqUCn/build/release/bin/kudu as pid 5835
I20260809 16:21:17.062412 21710 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786292417722232-21710-0/minicluster-data/ts-2/wal/instance
I20260809 16:21:17.062602 21710 external_mini_cluster.cc:1428] Setting key 19ed0cb3953e962f6053f0b199dbe372
Aug 09 16:21:17 dist-test-slave-s11t krb5kdc[5249](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292477, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.51.131@KRBTEST.COM for kudu/127.21.51.190@KRBTEST.COM
I20260809 16:21:17.067812 5369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:21:17.058922 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.51.131:52079 (local address 127.21.51.190:41021)
0809 16:21:17.058969 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0809 16:21:17.058971 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:21:17.059168 (+ 197us) server_negotiation.cc:407] Connection header received
0809 16:21:17.059649 (+ 481us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:21:17.059650 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:21:17.059660 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:21:17.059720 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:21:17.060643 (+ 923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:21:17.061219 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:21:17.062354 (+ 1135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:21:17.062716 (+ 362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:21:17.062823 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:21:17.065116 (+ 2293us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:21:17.065117 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:21:17.065117 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:21:17.065124 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:21:17.066488 (+ 1364us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:21:17.066925 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:21:17.066932 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:21:17.066932 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:21:17.066991 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:21:17.067262 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:21:17.067263 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:21:17.067264 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:21:17.067380 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:21:17.067434 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0809 16:21:17.067668 (+ 234us) server_negotiation.cc:299] Negotiation successful
0809 16:21:17.067738 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260809 16:21:17.067965 6047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:21:17.058877 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:41021 (local address 127.21.51.131:52079)
0809 16:21:17.059040 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0809 16:21:17.059081 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0809 16:21:17.059568 (+ 487us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:21:17.059806 (+ 238us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:21:17.059810 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:21:17.060371 (+ 561us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:21:17.060558 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:21:17.060563 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:21:17.061303 (+ 740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:21:17.061303 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:21:17.062236 (+ 933us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:21:17.062239 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:21:17.062851 (+ 612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:21:17.062852 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:21:17.063009 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:21:17.063539 (+ 530us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:21:17.063569 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:21:17.065029 (+ 1460us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:21:17.066583 (+ 1554us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:21:17.066589 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:21:17.066590 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:21:17.066844 (+ 254us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:21:17.067078 (+ 234us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:21:17.067078 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:21:17.067079 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:21:17.067193 (+ 114us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:21:17.067441 (+ 248us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:21:17.067445 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:21:17.067555 (+ 110us) client_negotiation.cc:769] Sending connection context
0809 16:21:17.067637 (+ 82us) client_negotiation.cc:240] Negotiation successful
0809 16:21:17.067756 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":93,"threads_started":1}
I20260809 16:21:17.068575 6042 heartbeater.cc:347] Connected to a master server at 127.21.51.190:41021
I20260809 16:21:17.068679 6042 heartbeater.cc:464] Registering TS with master...
I20260809 16:21:17.068871 6042 heartbeater.cc:511] Master 127.21.51.190:41021 requested a full tablet report, sending...
I20260809 16:21:17.069280 5314 ts_manager.cc:194] Registered new tserver with Master: a9be429d0ed24e5b86c8f333c1fbfb9f (127.21.51.131:33095)
I20260809 16:21:17.069619 5314 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.51.131@KRBTEST.COM'} at 127.21.51.131:52079
I20260809 16:21:17.074344 21710 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 16:21:17.081396 5369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:21:17.075617 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42060 (local address 127.21.51.190:41021)
0809 16:21:17.075674 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0809 16:21:17.075676 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:21:17.075770 (+ 94us) server_negotiation.cc:407] Connection header received
0809 16:21:17.075816 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:21:17.075817 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:21:17.075825 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:21:17.075874 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:21:17.076366 (+ 492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:21:17.076948 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:21:17.077645 (+ 697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:21:17.077826 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:21:17.077883 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:21:17.079024 (+ 1141us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:21:17.079025 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:21:17.079025 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:21:17.079031 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:21:17.080413 (+ 1382us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:21:17.080746 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:21:17.080747 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:21:17.080747 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:21:17.080798 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:21:17.080977 (+ 179us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:21:17.080977 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:21:17.080977 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:21:17.081109 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:21:17.081181 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0809 16:21:17.081266 (+ 85us) server_negotiation.cc:299] Negotiation successful
0809 16:21:17.081323 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260809 16:21:17.082973 5314 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42060:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 16:21:17.084539 5314 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 16:21:17.091166 6057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:21:17.086808 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.131:33095 (local address 127.0.0.1:56986)
0809 16:21:17.086924 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0809 16:21:17.086962 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0809 16:21:17.087028 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:21:17.087437 (+ 409us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:21:17.087440 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:21:17.087442 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:21:17.087531 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:21:17.087533 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:21:17.088956 (+ 1423us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:21:17.088957 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:21:17.089909 (+ 952us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:21:17.089912 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:21:17.090843 (+ 931us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:21:17.090843 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:21:17.090930 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:21:17.090934 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 16:21:17.090967 (+ 33us) client_negotiation.cc:240] Negotiation successful
0809 16:21:17.091054 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":92,"mutex_wait_us":4,"thread_start_us":63,"threads_started":1}
I20260809 16:21:17.091163 6064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:21:17.087078 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58810 (local address 127.21.51.130:36569)
0809 16:21:17.087279 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0809 16:21:17.087285 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 16:21:17.087517 (+ 232us) server_negotiation.cc:407] Connection header received
0809 16:21:17.087565 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:21:17.087567 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:21:17.087584 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:21:17.087660 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:21:17.087899 (+ 239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:21:17.088965 (+ 1066us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:21:17.090219 (+ 1254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:21:17.090702 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:21:17.090812 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:21:17.090873 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0809 16:21:17.091027 (+ 154us) server_negotiation.cc:299] Negotiation successful
0809 16:21:17.091072 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":163,"thread_start_us":102,"threads_started":1}
I20260809 16:21:17.091256 6065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:21:17.087297 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35180 (local address 127.21.51.129:37605)
0809 16:21:17.087517 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0809 16:21:17.087521 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 16:21:17.087532 (+ 11us) server_negotiation.cc:407] Connection header received
0809 16:21:17.087569 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:21:17.087570 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:21:17.087589 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:21:17.087660 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:21:17.087919 (+ 259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:21:17.088598 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:21:17.090035 (+ 1437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:21:17.090736 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:21:17.090811 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:21:17.090873 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0809 16:21:17.091127 (+ 254us) server_negotiation.cc:299] Negotiation successful
0809 16:21:17.091172 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":181,"thread_start_us":113,"threads_started":1}
I20260809 16:21:17.091320 6060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:21:17.087069 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56986 (local address 127.21.51.131:33095)
0809 16:21:17.087238 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0809 16:21:17.087244 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 16:21:17.087256 (+ 12us) server_negotiation.cc:407] Connection header received
0809 16:21:17.087296 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:21:17.087298 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:21:17.087315 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:21:17.087617 (+ 302us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:21:17.087682 (+ 65us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:21:17.088811 (+ 1129us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:21:17.090013 (+ 1202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:21:17.090736 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:21:17.091113 (+ 377us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:21:17.091173 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0809 16:21:17.091196 (+ 23us) server_negotiation.cc:299] Negotiation successful
0809 16:21:17.091235 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":131,"thread_start_us":87,"threads_started":1}
I20260809 16:21:17.091431 6059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:21:17.086829 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.130:36569 (local address 127.0.0.1:58810)
0809 16:21:17.087393 (+ 564us) negotiation.cc:107] Waiting for socket to connect
0809 16:21:17.087400 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0809 16:21:17.087460 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:21:17.087744 (+ 284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:21:17.087744 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:21:17.087747 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:21:17.087807 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:21:17.087809 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:21:17.089080 (+ 1271us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:21:17.089081 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:21:17.090121 (+ 1040us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:21:17.090123 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:21:17.090817 (+ 694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:21:17.090818 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:21:17.090900 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:21:17.090913 (+ 13us) client_negotiation.cc:769] Sending connection context
0809 16:21:17.091010 (+ 97us) client_negotiation.cc:240] Negotiation successful
0809 16:21:17.091080 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":557,"mutex_wait_us":25,"thread_start_us":54,"threads_started":1}
I20260809 16:21:17.091537 6058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:21:17.086814 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.129:37605 (local address 127.0.0.1:35180)
0809 16:21:17.087251 (+ 437us) negotiation.cc:107] Waiting for socket to connect
0809 16:21:17.087263 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 16:21:17.087326 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:21:17.087744 (+ 418us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:21:17.087744 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:21:17.087749 (+ 5us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:21:17.087837 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:21:17.087838 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:21:17.088736 (+ 898us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:21:17.088737 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:21:17.089941 (+ 1204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:21:17.089944 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:21:17.090903 (+ 959us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:21:17.090904 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:21:17.090955 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:21:17.090959 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 16:21:17.091004 (+ 45us) client_negotiation.cc:240] Negotiation successful
0809 16:21:17.091045 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":421,"thread_start_us":63,"threads_started":1}
I20260809 16:21:17.092391 5941 tablet_service.cc:1467] Processing CreateTablet for tablet f3a41f2fa58149cd801e2dc65e302413 (DEFAULT_TABLE table=test-table [id=4c5e914026ec4985bdc953c2899d0d3f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:21:17.092597 5506 tablet_service.cc:1467] Processing CreateTablet for tablet f3a41f2fa58149cd801e2dc65e302413 (DEFAULT_TABLE table=test-table [id=4c5e914026ec4985bdc953c2899d0d3f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:21:17.092589 5728 tablet_service.cc:1467] Processing CreateTablet for tablet f3a41f2fa58149cd801e2dc65e302413 (DEFAULT_TABLE table=test-table [id=4c5e914026ec4985bdc953c2899d0d3f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:21:17.092794 5941 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f3a41f2fa58149cd801e2dc65e302413. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:21:17.092948 5728 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f3a41f2fa58149cd801e2dc65e302413. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:21:17.092959 5506 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f3a41f2fa58149cd801e2dc65e302413. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:21:17.095505 6066 tablet_bootstrap.cc:492] T f3a41f2fa58149cd801e2dc65e302413 P a9be429d0ed24e5b86c8f333c1fbfb9f: Bootstrap starting.
I20260809 16:21:17.096203 6066 tablet_bootstrap.cc:654] T f3a41f2fa58149cd801e2dc65e302413 P a9be429d0ed24e5b86c8f333c1fbfb9f: Neither blocks nor log segments found. Creating new log.
I20260809 16:21:17.096354 6067 tablet_bootstrap.cc:492] T f3a41f2fa58149cd801e2dc65e302413 P 63fb26bc10de499389d505ec1d22f22c: Bootstrap starting.
I20260809 16:21:17.096730 6068 tablet_bootstrap.cc:492] T f3a41f2fa58149cd801e2dc65e302413 P df77827a888a44caab9d71371a152972: Bootstrap starting.
I20260809 16:21:17.096921 6066 log.cc:824] T f3a41f2fa58149cd801e2dc65e302413 P a9be429d0ed24e5b86c8f333c1fbfb9f: Log is configured to *not* fsync() on all Append() calls
I20260809 16:21:17.097082 6067 tablet_bootstrap.cc:654] T f3a41f2fa58149cd801e2dc65e302413 P 63fb26bc10de499389d505ec1d22f22c: Neither blocks nor log segments found. Creating new log.
I20260809 16:21:17.097412 6068 tablet_bootstrap.cc:654] T f3a41f2fa58149cd801e2dc65e302413 P df77827a888a44caab9d71371a152972: Neither blocks nor log segments found. Creating new log.
I20260809 16:21:17.097657 6067 log.cc:824] T f3a41f2fa58149cd801e2dc65e302413 P 63fb26bc10de499389d505ec1d22f22c: Log is configured to *not* fsync() on all Append() calls
I20260809 16:21:17.097834 6066 tablet_bootstrap.cc:492] T f3a41f2fa58149cd801e2dc65e302413 P a9be429d0ed24e5b86c8f333c1fbfb9f: No bootstrap required, opened a new log
I20260809 16:21:17.097908 6066 ts_tablet_manager.cc:1397] T f3a41f2fa58149cd801e2dc65e302413 P a9be429d0ed24e5b86c8f333c1fbfb9f: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 16:21:17.098030 6068 log.cc:824] T f3a41f2fa58149cd801e2dc65e302413 P df77827a888a44caab9d71371a152972: Log is configured to *not* fsync() on all Append() calls
I20260809 16:21:17.098409 6067 tablet_bootstrap.cc:492] T f3a41f2fa58149cd801e2dc65e302413 P 63fb26bc10de499389d505ec1d22f22c: No bootstrap required, opened a new log
I20260809 16:21:17.098475 6067 ts_tablet_manager.cc:1397] T f3a41f2fa58149cd801e2dc65e302413 P 63fb26bc10de499389d505ec1d22f22c: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260809 16:21:17.099422 6066 raft_consensus.cc:348] T f3a41f2fa58149cd801e2dc65e302413 P a9be429d0ed24e5b86c8f333c1fbfb9f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df77827a888a44caab9d71371a152972" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 37605 } } peers { permanent_uuid: "a9be429d0ed24e5b86c8f333c1fbfb9f" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 33095 } } peers { permanent_uuid: "63fb26bc10de499389d505ec1d22f22c" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 36569 } }
I20260809 16:21:17.099538 6066 raft_consensus.cc:374] T f3a41f2fa58149cd801e2dc65e302413 P a9be429d0ed24e5b86c8f333c1fbfb9f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:21:17.099558 6066 raft_consensus.cc:729] T f3a41f2fa58149cd801e2dc65e302413 P a9be429d0ed24e5b86c8f333c1fbfb9f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a9be429d0ed24e5b86c8f333c1fbfb9f, State: Initialized, Role: FOLLOWER
I20260809 16:21:17.099675 6066 consensus_queue.cc:260] T f3a41f2fa58149cd801e2dc65e302413 P a9be429d0ed24e5b86c8f333c1fbfb9f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df77827a888a44caab9d71371a152972" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 37605 } } peers { permanent_uuid: "a9be429d0ed24e5b86c8f333c1fbfb9f" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 33095 } } peers { permanent_uuid: "63fb26bc10de499389d505ec1d22f22c" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 36569 } }
I20260809 16:21:17.099756 6067 raft_consensus.cc:348] T f3a41f2fa58149cd801e2dc65e302413 P 63fb26bc10de499389d505ec1d22f22c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df77827a888a44caab9d71371a152972" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 37605 } } peers { permanent_uuid: "a9be429d0ed24e5b86c8f333c1fbfb9f" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 33095 } } peers { permanent_uuid: "63fb26bc10de499389d505ec1d22f22c" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 36569 } }
I20260809 16:21:17.099912 6066 ts_tablet_manager.cc:1428] T f3a41f2fa58149cd801e2dc65e302413 P a9be429d0ed24e5b86c8f333c1fbfb9f: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260809 16:21:17.099918 6067 raft_consensus.cc:374] T f3a41f2fa58149cd801e2dc65e302413 P 63fb26bc10de499389d505ec1d22f22c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:21:17.099965 6067 raft_consensus.cc:729] T f3a41f2fa58149cd801e2dc65e302413 P 63fb26bc10de499389d505ec1d22f22c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 63fb26bc10de499389d505ec1d22f22c, State: Initialized, Role: FOLLOWER
I20260809 16:21:17.100018 6042 heartbeater.cc:503] Master 127.21.51.190:41021 was elected leader, sending a full tablet report...
I20260809 16:21:17.100095 6067 consensus_queue.cc:260] T f3a41f2fa58149cd801e2dc65e302413 P 63fb26bc10de499389d505ec1d22f22c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df77827a888a44caab9d71371a152972" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 37605 } } peers { permanent_uuid: "a9be429d0ed24e5b86c8f333c1fbfb9f" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 33095 } } peers { permanent_uuid: "63fb26bc10de499389d505ec1d22f22c" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 36569 } }
I20260809 16:21:17.100419 5830 heartbeater.cc:503] Master 127.21.51.190:41021 was elected leader, sending a full tablet report...
I20260809 16:21:17.100575 6067 ts_tablet_manager.cc:1428] T f3a41f2fa58149cd801e2dc65e302413 P 63fb26bc10de499389d505ec1d22f22c: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260809 16:21:17.101261 6068 tablet_bootstrap.cc:492] T f3a41f2fa58149cd801e2dc65e302413 P df77827a888a44caab9d71371a152972: No bootstrap required, opened a new log
I20260809 16:21:17.101315 6068 ts_tablet_manager.cc:1397] T f3a41f2fa58149cd801e2dc65e302413 P df77827a888a44caab9d71371a152972: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260809 16:21:17.102720 6068 raft_consensus.cc:348] T f3a41f2fa58149cd801e2dc65e302413 P df77827a888a44caab9d71371a152972 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df77827a888a44caab9d71371a152972" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 37605 } } peers { permanent_uuid: "a9be429d0ed24e5b86c8f333c1fbfb9f" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 33095 } } peers { permanent_uuid: "63fb26bc10de499389d505ec1d22f22c" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 36569 } }
I20260809 16:21:17.102840 6068 raft_consensus.cc:374] T f3a41f2fa58149cd801e2dc65e302413 P df77827a888a44caab9d71371a152972 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:21:17.102861 6068 raft_consensus.cc:729] T f3a41f2fa58149cd801e2dc65e302413 P df77827a888a44caab9d71371a152972 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: df77827a888a44caab9d71371a152972, State: Initialized, Role: FOLLOWER
I20260809 16:21:17.102999 6068 consensus_queue.cc:260] T f3a41f2fa58149cd801e2dc65e302413 P df77827a888a44caab9d71371a152972 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df77827a888a44caab9d71371a152972" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 37605 } } peers { permanent_uuid: "a9be429d0ed24e5b86c8f333c1fbfb9f" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 33095 } } peers { permanent_uuid: "63fb26bc10de499389d505ec1d22f22c" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 36569 } }
I20260809 16:21:17.103248 6068 ts_tablet_manager.cc:1428] T f3a41f2fa58149cd801e2dc65e302413 P df77827a888a44caab9d71371a152972: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260809 16:21:17.103313 5613 heartbeater.cc:503] Master 127.21.51.190:41021 was elected leader, sending a full tablet report...
I20260809 16:21:17.128857 6076 raft_consensus.cc:482] T f3a41f2fa58149cd801e2dc65e302413 P 63fb26bc10de499389d505ec1d22f22c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 16:21:17.128971 6076 raft_consensus.cc:504] T f3a41f2fa58149cd801e2dc65e302413 P 63fb26bc10de499389d505ec1d22f22c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df77827a888a44caab9d71371a152972" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 37605 } } peers { permanent_uuid: "a9be429d0ed24e5b86c8f333c1fbfb9f" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 33095 } } peers { permanent_uuid: "63fb26bc10de499389d505ec1d22f22c" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 36569 } }
I20260809 16:21:17.129273 6076 leader_election.cc:288] T f3a41f2fa58149cd801e2dc65e302413 P 63fb26bc10de499389d505ec1d22f22c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers df77827a888a44caab9d71371a152972 (127.21.51.129:37605), a9be429d0ed24e5b86c8f333c1fbfb9f (127.21.51.131:33095)
I20260809 16:21:17.132732 6060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:21:17.129574 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.51.130:34997 (local address 127.21.51.131:33095)
0809 16:21:17.129627 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0809 16:21:17.129629 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:21:17.129892 (+ 263us) server_negotiation.cc:407] Connection header received
0809 16:21:17.129948 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:21:17.129949 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:21:17.129959 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:21:17.130036 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:21:17.130205 (+ 169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:21:17.130736 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:21:17.131784 (+ 1048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:21:17.132337 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:21:17.132438 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:21:17.132447 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0809 16:21:17.132610 (+ 163us) server_negotiation.cc:299] Negotiation successful
0809 16:21:17.132644 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260809 16:21:17.132824 6065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:21:17.129542 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.51.130:35733 (local address 127.21.51.129:37605)
0809 16:21:17.129604 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0809 16:21:17.129607 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 16:21:17.129617 (+ 10us) server_negotiation.cc:407] Connection header received
0809 16:21:17.129689 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:21:17.129690 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:21:17.129698 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:21:17.129775 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:21:17.129956 (+ 181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:21:17.130620 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:21:17.131838 (+ 1218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:21:17.132396 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:21:17.132490 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:21:17.132499 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0809 16:21:17.132705 (+ 206us) server_negotiation.cc:299] Negotiation successful
0809 16:21:17.132747 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260809 16:21:17.132850 5832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:21:17.129472 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.129:37605 (local address 127.21.51.130:35733)
0809 16:21:17.129564 (+ 92us) negotiation.cc:107] Waiting for socket to connect
0809 16:21:17.129577 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 16:21:17.129626 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:21:17.129789 (+ 163us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:21:17.129790 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:21:17.129792 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:21:17.129869 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:21:17.129870 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:21:17.130732 (+ 862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:21:17.130733 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:21:17.131686 (+ 953us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:21:17.131689 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:21:17.132501 (+ 812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:21:17.132502 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:21:17.132594 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:21:17.132597 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 16:21:17.132670 (+ 73us) client_negotiation.cc:240] Negotiation successful
0809 16:21:17.132744 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":71}
I20260809 16:21:17.132746 6078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:21:17.129472 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.131:33095 (local address 127.21.51.130:34997)
0809 16:21:17.129814 (+ 342us) negotiation.cc:107] Waiting for socket to connect
0809 16:21:17.129825 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 16:21:17.129894 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:21:17.130042 (+ 148us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:21:17.130043 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:21:17.130044 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:21:17.130113 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:21:17.130114 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:21:17.130828 (+ 714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:21:17.130829 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:21:17.131699 (+ 870us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:21:17.131701 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:21:17.132440 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:21:17.132441 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:21:17.132517 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:21:17.132527 (+ 10us) client_negotiation.cc:769] Sending connection context
0809 16:21:17.132601 (+ 74us) client_negotiation.cc:240] Negotiation successful
0809 16:21:17.132641 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":320,"mutex_wait_us":100,"thread_start_us":165,"threads_started":1}
I20260809 16:21:17.133244 5536 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f3a41f2fa58149cd801e2dc65e302413" candidate_uuid: "63fb26bc10de499389d505ec1d22f22c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "df77827a888a44caab9d71371a152972" is_pre_election: true
I20260809 16:21:17.133407 5536 raft_consensus.cc:2454] T f3a41f2fa58149cd801e2dc65e302413 P df77827a888a44caab9d71371a152972 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 63fb26bc10de499389d505ec1d22f22c in term 0.
I20260809 16:21:17.133440 5970 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f3a41f2fa58149cd801e2dc65e302413" candidate_uuid: "63fb26bc10de499389d505ec1d22f22c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a9be429d0ed24e5b86c8f333c1fbfb9f" is_pre_election: true
I20260809 16:21:17.133598 5970 raft_consensus.cc:2454] T f3a41f2fa58149cd801e2dc65e302413 P a9be429d0ed24e5b86c8f333c1fbfb9f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 63fb26bc10de499389d505ec1d22f22c in term 0.
I20260809 16:21:17.133677 5650 leader_election.cc:302] T f3a41f2fa58149cd801e2dc65e302413 P 63fb26bc10de499389d505ec1d22f22c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 63fb26bc10de499389d505ec1d22f22c, df77827a888a44caab9d71371a152972; no voters:
I20260809 16:21:17.133816 6076 raft_consensus.cc:2781] T f3a41f2fa58149cd801e2dc65e302413 P 63fb26bc10de499389d505ec1d22f22c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 16:21:17.133862 6076 raft_consensus.cc:482] T f3a41f2fa58149cd801e2dc65e302413 P 63fb26bc10de499389d505ec1d22f22c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 16:21:17.133889 6076 raft_consensus.cc:3037] T f3a41f2fa58149cd801e2dc65e302413 P 63fb26bc10de499389d505ec1d22f22c [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:21:17.134835 6076 raft_consensus.cc:504] T f3a41f2fa58149cd801e2dc65e302413 P 63fb26bc10de499389d505ec1d22f22c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df77827a888a44caab9d71371a152972" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 37605 } } peers { permanent_uuid: "a9be429d0ed24e5b86c8f333c1fbfb9f" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 33095 } } peers { permanent_uuid: "63fb26bc10de499389d505ec1d22f22c" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 36569 } }
I20260809 16:21:17.135025 6076 leader_election.cc:288] T f3a41f2fa58149cd801e2dc65e302413 P 63fb26bc10de499389d505ec1d22f22c [CANDIDATE]: Term 1 election: Requested vote from peers df77827a888a44caab9d71371a152972 (127.21.51.129:37605), a9be429d0ed24e5b86c8f333c1fbfb9f (127.21.51.131:33095)
I20260809 16:21:17.135206 5970 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f3a41f2fa58149cd801e2dc65e302413" candidate_uuid: "63fb26bc10de499389d505ec1d22f22c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a9be429d0ed24e5b86c8f333c1fbfb9f"
I20260809 16:21:17.135290 5970 raft_consensus.cc:3037] T f3a41f2fa58149cd801e2dc65e302413 P a9be429d0ed24e5b86c8f333c1fbfb9f [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:21:17.135272 5536 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f3a41f2fa58149cd801e2dc65e302413" candidate_uuid: "63fb26bc10de499389d505ec1d22f22c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "df77827a888a44caab9d71371a152972"
I20260809 16:21:17.135353 5536 raft_consensus.cc:3037] T f3a41f2fa58149cd801e2dc65e302413 P df77827a888a44caab9d71371a152972 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:21:17.136027 5970 raft_consensus.cc:2454] T f3a41f2fa58149cd801e2dc65e302413 P a9be429d0ed24e5b86c8f333c1fbfb9f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 63fb26bc10de499389d505ec1d22f22c in term 1.
I20260809 16:21:17.136193 5536 raft_consensus.cc:2454] T f3a41f2fa58149cd801e2dc65e302413 P df77827a888a44caab9d71371a152972 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 63fb26bc10de499389d505ec1d22f22c in term 1.
I20260809 16:21:17.136279 5654 leader_election.cc:302] T f3a41f2fa58149cd801e2dc65e302413 P 63fb26bc10de499389d505ec1d22f22c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 63fb26bc10de499389d505ec1d22f22c, a9be429d0ed24e5b86c8f333c1fbfb9f; no voters:
I20260809 16:21:17.136374 6076 raft_consensus.cc:2781] T f3a41f2fa58149cd801e2dc65e302413 P 63fb26bc10de499389d505ec1d22f22c [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:21:17.136584 6076 raft_consensus.cc:686] T f3a41f2fa58149cd801e2dc65e302413 P 63fb26bc10de499389d505ec1d22f22c [term 1 LEADER]: Becoming Leader. State: Replica: 63fb26bc10de499389d505ec1d22f22c, State: Running, Role: LEADER
I20260809 16:21:17.136727 6076 consensus_queue.cc:237] T f3a41f2fa58149cd801e2dc65e302413 P 63fb26bc10de499389d505ec1d22f22c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df77827a888a44caab9d71371a152972" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 37605 } } peers { permanent_uuid: "a9be429d0ed24e5b86c8f333c1fbfb9f" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 33095 } } peers { permanent_uuid: "63fb26bc10de499389d505ec1d22f22c" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 36569 } }
I20260809 16:21:17.137687 5314 catalog_manager.cc:5495] T f3a41f2fa58149cd801e2dc65e302413 P 63fb26bc10de499389d505ec1d22f22c reported cstate change: term changed from 0 to 1, leader changed from <none> to 63fb26bc10de499389d505ec1d22f22c (127.21.51.130). New cstate: current_term: 1 leader_uuid: "63fb26bc10de499389d505ec1d22f22c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df77827a888a44caab9d71371a152972" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 37605 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a9be429d0ed24e5b86c8f333c1fbfb9f" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 33095 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "63fb26bc10de499389d505ec1d22f22c" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 36569 } health_report { overall_health: HEALTHY } } }
W20260809 16:21:17.143569 5831 tablet.cc:2369] T f3a41f2fa58149cd801e2dc65e302413 P 63fb26bc10de499389d505ec1d22f22c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 16:21:17.148257 6064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:21:17.145792 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58814 (local address 127.21.51.130:36569)
0809 16:21:17.145854 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0809 16:21:17.145857 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 16:21:17.145869 (+ 12us) server_negotiation.cc:407] Connection header received
0809 16:21:17.145899 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:21:17.145900 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:21:17.145916 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:21:17.145988 (+ 72us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 16:21:17.146140 (+ 152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:21:17.146593 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:21:17.147424 (+ 831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:21:17.147632 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:21:17.147708 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:21:17.147923 (+ 215us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 16:21:17.148027 (+ 104us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 16:21:17.148126 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0809 16:21:17.148145 (+ 19us) server_negotiation.cc:299] Negotiation successful
0809 16:21:17.148165 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260809 16:21:17.150213 5970 raft_consensus.cc:1261] T f3a41f2fa58149cd801e2dc65e302413 P a9be429d0ed24e5b86c8f333c1fbfb9f [term 1 FOLLOWER]: Refusing update from remote peer 63fb26bc10de499389d505ec1d22f22c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 16:21:17.150261 5536 raft_consensus.cc:1261] T f3a41f2fa58149cd801e2dc65e302413 P df77827a888a44caab9d71371a152972 [term 1 FOLLOWER]: Refusing update from remote peer 63fb26bc10de499389d505ec1d22f22c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 16:21:17.150513 6076 consensus_queue.cc:1035] T f3a41f2fa58149cd801e2dc65e302413 P 63fb26bc10de499389d505ec1d22f22c [LEADER]: Connected to new peer: Peer: permanent_uuid: "a9be429d0ed24e5b86c8f333c1fbfb9f" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 33095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 16:21:17.150648 6083 consensus_queue.cc:1035] T f3a41f2fa58149cd801e2dc65e302413 P 63fb26bc10de499389d505ec1d22f22c [LEADER]: Connected to new peer: Peer: permanent_uuid: "df77827a888a44caab9d71371a152972" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 37605 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 16:21:17.153065 6086 mvcc.cc:205] Tried to move back new op lower bound from 7316653986404384768 to 7316653986352967680. Current Snapshot: MvccSnapshot[applied={T|T < 7316653986404384768}]
I20260809 16:21:17.155187 5313 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42060:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 16:21:17.155287 5313 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42060:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 16:21:17.156054 5313 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4a24add1b1cc4881ae9579132b4c8dec: Sending DeleteTablet for 3 replicas of tablet f3a41f2fa58149cd801e2dc65e302413
I20260809 16:21:17.156431 5941 tablet_service.cc:1514] Processing DeleteTablet for tablet f3a41f2fa58149cd801e2dc65e302413 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:21:17 UTC) from {username='kudu', principal='kudu/127.21.51.190@KRBTEST.COM'} at 127.0.0.1:56986
I20260809 16:21:17.156431 5506 tablet_service.cc:1514] Processing DeleteTablet for tablet f3a41f2fa58149cd801e2dc65e302413 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:21:17 UTC) from {username='kudu', principal='kudu/127.21.51.190@KRBTEST.COM'} at 127.0.0.1:35180
I20260809 16:21:17.156862 6103 tablet_replica.cc:330] T f3a41f2fa58149cd801e2dc65e302413 P a9be429d0ed24e5b86c8f333c1fbfb9f: stopping tablet replica
I20260809 16:21:17.156834 5728 tablet_service.cc:1514] Processing DeleteTablet for tablet f3a41f2fa58149cd801e2dc65e302413 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:21:17 UTC) from {username='kudu', principal='kudu/127.21.51.190@KRBTEST.COM'} at 127.0.0.1:58810
I20260809 16:21:17.157016 6101 tablet_replica.cc:330] T f3a41f2fa58149cd801e2dc65e302413 P df77827a888a44caab9d71371a152972: stopping tablet replica
I20260809 16:21:17.157155 6101 raft_consensus.cc:2229] T f3a41f2fa58149cd801e2dc65e302413 P df77827a888a44caab9d71371a152972 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 16:21:17.156648 21710 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrNqUCn/build/release/bin/kudu with pid 5411
I20260809 16:21:17.157280 6101 raft_consensus.cc:2258] T f3a41f2fa58149cd801e2dc65e302413 P df77827a888a44caab9d71371a152972 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 16:21:17.157419 6103 raft_consensus.cc:2229] T f3a41f2fa58149cd801e2dc65e302413 P a9be429d0ed24e5b86c8f333c1fbfb9f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 16:21:17.157502 6103 raft_consensus.cc:2258] T f3a41f2fa58149cd801e2dc65e302413 P a9be429d0ed24e5b86c8f333c1fbfb9f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 16:21:17.165015 6103 ts_tablet_manager.cc:1905] T f3a41f2fa58149cd801e2dc65e302413 P a9be429d0ed24e5b86c8f333c1fbfb9f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 16:21:17.167091 6103 ts_tablet_manager.cc:1918] T f3a41f2fa58149cd801e2dc65e302413 P a9be429d0ed24e5b86c8f333c1fbfb9f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 16:21:17.167194 6103 log.cc:1196] T f3a41f2fa58149cd801e2dc65e302413 P a9be429d0ed24e5b86c8f333c1fbfb9f: Deleting WAL directory at /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786292417722232-21710-0/minicluster-data/ts-2/wal/wals/f3a41f2fa58149cd801e2dc65e302413
I20260809 16:21:17.167400 6104 tablet_replica.cc:330] T f3a41f2fa58149cd801e2dc65e302413 P 63fb26bc10de499389d505ec1d22f22c: stopping tablet replica
I20260809 16:21:17.167527 6103 ts_tablet_manager.cc:1939] T f3a41f2fa58149cd801e2dc65e302413 P a9be429d0ed24e5b86c8f333c1fbfb9f: Deleting consensus metadata
I20260809 16:21:17.167567 6104 raft_consensus.cc:2229] T f3a41f2fa58149cd801e2dc65e302413 P 63fb26bc10de499389d505ec1d22f22c [term 1 LEADER]: Raft consensus shutting down.
I20260809 16:21:17.167769 6104 raft_consensus.cc:2258] T f3a41f2fa58149cd801e2dc65e302413 P 63fb26bc10de499389d505ec1d22f22c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 16:21:17.167984 5298 catalog_manager.cc:4841] TS a9be429d0ed24e5b86c8f333c1fbfb9f (127.21.51.131:33095): tablet f3a41f2fa58149cd801e2dc65e302413 (table test-table [id=4c5e914026ec4985bdc953c2899d0d3f]) successfully deleted
I20260809 16:21:17.168340 6104 ts_tablet_manager.cc:1905] T f3a41f2fa58149cd801e2dc65e302413 P 63fb26bc10de499389d505ec1d22f22c: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260809 16:21:17.169757 5292 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.21.51.129:37605 (error 108)
I20260809 16:21:17.170264 6104 ts_tablet_manager.cc:1918] T f3a41f2fa58149cd801e2dc65e302413 P 63fb26bc10de499389d505ec1d22f22c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 16:21:17.170346 6104 log.cc:1196] T f3a41f2fa58149cd801e2dc65e302413 P 63fb26bc10de499389d505ec1d22f22c: Deleting WAL directory at /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786292417722232-21710-0/minicluster-data/ts-1/wal/wals/f3a41f2fa58149cd801e2dc65e302413
I20260809 16:21:17.170615 21710 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrNqUCn/build/release/bin/kudu with pid 5621
I20260809 16:21:17.170632 6058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:21:17.170470 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.129:37605 (local address 127.0.0.1:35192)
0809 16:21:17.170539 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0809 16:21:17.170595 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.51.129:37605: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
I20260809 16:21:17.170660 6104 ts_tablet_manager.cc:1939] T f3a41f2fa58149cd801e2dc65e302413 P 63fb26bc10de499389d505ec1d22f22c: Deleting consensus metadata
W20260809 16:21:17.170747 5292 catalog_manager.cc:4584] TS df77827a888a44caab9d71371a152972 (127.21.51.129:37605): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet f3a41f2fa58149cd801e2dc65e302413: Network error: Client connection negotiation failed: client connection to 127.21.51.129:37605: connect: Connection refused (error 111)
I20260809 16:21:17.177635 6058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:21:17.177493 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.130:36569 (local address 127.0.0.1:58820)
0809 16:21:17.177545 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0809 16:21:17.177598 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.51.130:36569: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260809 16:21:17.177647 21710 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrNqUCn/build/release/bin/kudu with pid 5835
I20260809 16:21:17.191368 21710 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrNqUCn/build/release/bin/kudu with pid 5265
2026-08-09T16:21:17Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (2134 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786292417722232-21710-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 16:21:17 dist-test-slave-s11t krb5kdc[6109](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 16:21:17 dist-test-slave-s11t krb5kdc[6109](info): set up 2 sockets
Aug 09 16:21:17 dist-test-slave-s11t krb5kdc[6109](info): commencing operation
krb5kdc: starting...
W20260809 16:21:18.431277 21710 mini_kdc.cc:121] Time spent starting KDC: real 1.216s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786292417722232-21710-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786292417722232-21710-0/krb5kdc/test-user.keytab.
Aug 09 16:21:18 dist-test-slave-s11t krb5kdc[6109](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292478, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T16:21:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T16:21:18Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.21.51.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.21.51.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.21.51.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786292417722232-21710-0/krb5kdc/rangeradmin_127.21.51.148@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.21.51.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786292417722232-21710-0/krb5kdc/rangeradmin_127.21.51.148@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.21.51.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.21.51.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.21.51.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786292417722232-21710-0/krb5kdc/rangerlookup_127.21.51.148@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.21.51.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786292417722232-21710-0/krb5kdc/rangerlookup_127.21.51.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.21.51.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.21.51.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.21.51.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786292417722232-21710-0/krb5kdc/HTTP_127.21.51.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.21.51.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786292417722232-21710-0/krb5kdc/HTTP_127.21.51.148@KRBTEST.COM.keytab.
I20260809 16:21:18.607438 21710 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786292417722232-21710-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskrNqUCn/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786292417722232-21710-0/minicluster-data/postgres -l logfile start
2026-08-09 16:21:21.081 UTC [6160] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-09 16:21:21.081 UTC [6160] LOG: listening on IPv4 address "127.21.51.148", port 52187
2026-08-09 16:21:21.084 UTC [6160] LOG: listening on Unix socket "/tmp/.s.PGSQL.52187"
2026-08-09 16:21:21.121 UTC [6163] LOG: database system was shut down at 2026-08-09 16:21:19 UTC
2026-08-09 16:21:21.125 UTC [6160] LOG: database system is ready to accept connections
I20260809 16:21:22.289754 21710 mini_postgres.cc:96] Postgres bound to 52187
2026-08-09 16:21:22.297 UTC [6175] FATAL: database "slave" does not exist
127.21.51.148:52187 - accepting connections
I20260809 16:21:22.298087 21710 mini_ranger.cc:161] Starting Ranger...
I20260809 16:21:22.313700 21710 mini_ranger.cc:84] Created miniranger Postgres user
I20260809 16:21:22.756811 21710 mini_ranger.cc:87] Created ranger Postgres database
I20260809 16:21:22.756886 21710 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786292417722232-21710-0/minicluster-data/ranger-admin
2026-08-09 16:21:23,046 [I] DB FLAVOR :POSTGRES
2026-08-09 16:21:23,047 [I] --------- Verifying Ranger DB connection ---------
2026-08-09 16:21:23,047 [I] Checking connection..
2026-08-09 16:21:23,047 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrNqUCn/build/release/bin/postgresql.jar:/tmp/dist-test-taskrNqUCn/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.51.148:52187/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-09 16:21:23,304 [I] Checking connection passed.
2026-08-09 16:21:23,304 [I] --------- Verifying version history table ---------
2026-08-09 16:21:23,304 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrNqUCn/build/release/bin/postgresql.jar:/tmp/dist-test-taskrNqUCn/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.51.148:52187/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-09 16:21:23,562 [I] Table x_db_version_h does not exist in database ranger
2026-08-09 16:21:23,562 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrNqUCn/build/release/bin/postgresql.jar:/tmp/dist-test-taskrNqUCn/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.51.148:52187/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-09 16:21:23,829 [I] Table x_db_version_h does not exist in database ranger
2026-08-09 16:21:23,829 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-09 16:21:23,829 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrNqUCn/build/release/bin/postgresql.jar:/tmp/dist-test-taskrNqUCn/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.51.148:52187/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskrNqUCn/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-09 16:21:24.088 UTC [6271] WARNING: there is no transaction in progress
2026-08-09 16:21:24,095 [I] create_dbversion_catalog.sql file imported successfully
2026-08-09 16:21:24,096 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrNqUCn/build/release/bin/postgresql.jar:/tmp/dist-test-taskrNqUCn/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.51.148:52187/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-09 16:21:24,362 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-09 16:21:24,362 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-09 16:21:24,362 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrNqUCn/build/release/bin/postgresql.jar:/tmp/dist-test-taskrNqUCn/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.51.148:52187/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-09 16:21:24,593 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrNqUCn/build/release/bin/postgresql.jar:/tmp/dist-test-taskrNqUCn/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.51.148:52187/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-09 16:21:24,852 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrNqUCn/build/release/bin/postgresql.jar:/tmp/dist-test-taskrNqUCn/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.51.148:52187/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-s11t.c.gcp-upstream.internal','N') ;"
2026-08-09 16:21:25,096 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrNqUCn/build/release/bin/postgresql.jar:/tmp/dist-test-taskrNqUCn/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.51.148:52187/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-09 16:21:25,354 [I] Table x_portal_user does not exist in database ranger
2026-08-09 16:21:25,355 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrNqUCn/build/release/bin/postgresql.jar:/tmp/dist-test-taskrNqUCn/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.51.148:52187/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-09 16:21:25,598 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-09 16:21:25,599 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrNqUCn/build/release/bin/postgresql.jar:/tmp/dist-test-taskrNqUCn/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.51.148:52187/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-09 16:21:25,854 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrNqUCn/build/release/bin/postgresql.jar:/tmp/dist-test-taskrNqUCn/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.51.148:52187/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-09 16:21:26,102 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-09 16:21:26,103 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrNqUCn/build/release/bin/postgresql.jar:/tmp/dist-test-taskrNqUCn/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.51.148:52187/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskrNqUCn/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-09 16:21:27.068 UTC [6427] WARNING: there is no transaction in progress
2026-08-09 16:21:27.085 UTC [6427] WARNING: there is no transaction in progress
2026-08-09 16:21:27.100 UTC [6427] WARNING: there is no transaction in progress
2026-08-09 16:21:27.114 UTC [6427] WARNING: there is no transaction in progress
2026-08-09 16:21:27.128 UTC [6427] WARNING: there is no transaction in progress
2026-08-09 16:21:27.145 UTC [6427] WARNING: there is no transaction in progress
2026-08-09 16:21:27.197 UTC [6427] WARNING: there is no transaction in progress
2026-08-09 16:21:27.204 UTC [6427] WARNING: there is no transaction in progress
2026-08-09 16:21:27.218 UTC [6427] WARNING: there is no transaction in progress
2026-08-09 16:21:27.228 UTC [6427] WARNING: there is no transaction in progress
2026-08-09 16:21:27.239 UTC [6427] WARNING: there is no transaction in progress
2026-08-09 16:21:27.252 UTC [6427] WARNING: there is no transaction in progress
2026-08-09 16:21:27.262 UTC [6427] WARNING: there is no transaction in progress
2026-08-09 16:21:27.269 UTC [6427] WARNING: there is no transaction in progress
2026-08-09 16:21:27.276 UTC [6427] WARNING: there is no transaction in progress
2026-08-09 16:21:27.282 UTC [6427] WARNING: there is no transaction in progress
2026-08-09 16:21:27,892 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-09 16:21:27,893 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrNqUCn/build/release/bin/postgresql.jar:/tmp/dist-test-taskrNqUCn/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.51.148:52187/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-09 16:21:28,138 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-09 16:21:28,138 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrNqUCn/build/release/bin/postgresql.jar:/tmp/dist-test-taskrNqUCn/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.51.148:52187/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-09 16:21:28,400 [I] Table x_portal_user already exists in database 'ranger'
2026-08-09 16:21:28,401 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrNqUCn/build/release/bin/postgresql.jar:/tmp/dist-test-taskrNqUCn/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.51.148:52187/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-09 16:21:28,670 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-09 16:21:28,670 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrNqUCn/build/release/bin/postgresql.jar:/tmp/dist-test-taskrNqUCn/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.51.148:52187/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-09 16:21:28,935 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrNqUCn/build/release/bin/postgresql.jar:/tmp/dist-test-taskrNqUCn/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.51.148:52187/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-09 16:21:29,203 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrNqUCn/build/release/bin/postgresql.jar:/tmp/dist-test-taskrNqUCn/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.51.148:52187/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-s11t.c.gcp-upstream.internal';"
2026-08-09 16:21:29,447 [I] CORE_DB_SCHEMA import status has been updated
2026-08-09 16:21:29,448 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrNqUCn/build/release/bin/postgresql.jar:/tmp/dist-test-taskrNqUCn/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.51.148:52187/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-09 16:21:29,712 [I] DB_PATCHES have already been applied
I20260809 16:21:29.718593 21710 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786292417722232-21710-0/minicluster-data/ranger-admin:/tmp/dist-test-taskrNqUCn/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskrNqUCn/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskrNqUCn/build/release/bin/postgresql.jar:/tmp/dist-test-taskrNqUCn/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260809 16:21:29.718695 21710 mini_ranger.cc:193] Using host: 127.21.51.148
I20260809 16:21:29.722584 21710 mini_ranger.cc:227] Ranger admin URL: http://127.21.51.148:50097
Aug 09, 2026 4:21:30 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 09, 2026 4:21:30 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 09, 2026 4:21:30 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786292417722232-21710-0/minicluster-data/ranger-admin/ews/webapp
Aug 09, 2026 4:21:30 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786292417722232-21710-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 09, 2026 4:21:30 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786292417722232-21710-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 09, 2026 4:21:30 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786292417722232-21710-0/minicluster-data/ranger-admin/ews/webapp].
Aug 09, 2026 4:21:30 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260809 16:21:31.044919 21710 mini_ranger.cc:160] Time spent starting Ranger: real 8.747s user 0.001s sys 0.010s
Aug 09, 2026 4:21:51 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 09, 2026 4:21:51 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.MetricsREST
Aug 09, 2026 4:21:51 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 09, 2026 4:21:51 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 09, 2026 4:21:51 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 09, 2026 4:21:51 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 09, 2026 4:21:51 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 09, 2026 4:21:51 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 09, 2026 4:21:51 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 09, 2026 4:21:51 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 09, 2026 4:21:51 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 09, 2026 4:21:51 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 09, 2026 4:21:51 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 09, 2026 4:21:51 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 09, 2026 4:21:51 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 09, 2026 4:21:51 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 09, 2026 4:21:51 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 09, 2026 4:21:52 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260809 16:21:53.242380 21710 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.21.51.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.21.51.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.21.51.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786292417722232-21710-0/krb5kdc/rangerkms_127.21.51.148@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.21.51.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786292417722232-21710-0/krb5kdc/rangerkms_127.21.51.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.21.51.148@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.21.51.148@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.21.51.148@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786292417722232-21710-0/krb5kdc/HTTP_127.21.51.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.21.51.148@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786292417722232-21710-0/krb5kdc/HTTP_127.21.51.148@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 09 16:21:53 dist-test-slave-s11t krb5kdc[6109](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292513, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260809 16:21:53.321339 21710 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260809 16:21:53.330786 21710 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260809 16:21:54.120134 21710 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260809 16:21:54.120229 21710 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786292417722232-21710-0/minicluster-data/ranger-kms
I20260809 16:21:54.120252 21710 mini_ranger_kms.cc:227] Using postgres at 127.21.51.148:52187
2026-08-09 16:21:54,455 [I] DB FLAVOR :POSTGRES
2026-08-09 16:21:54,455 [I] --------- Verifying Ranger DB connection ---------
2026-08-09 16:21:54,455 [I] Checking connection
2026-08-09 16:21:54,455 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrNqUCn/build/release/bin/postgresql.jar:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786292417722232-21710-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.51.148:52187/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-09 16:21:54,696 [I] connection success
2026-08-09 16:21:54,696 [I] --------- Verifying Ranger DB tables ---------
2026-08-09 16:21:54,696 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-09 16:21:54,696 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrNqUCn/build/release/bin/postgresql.jar:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786292417722232-21710-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.51.148:52187/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-09 16:21:54,946 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-09 16:21:54,946 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-09 16:21:54,946 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-09 16:21:54,946 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrNqUCn/build/release/bin/postgresql.jar:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786292417722232-21710-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.51.148:52187/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786292417722232-21710-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-09 16:21:55,202 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260809 16:21:55.655747 21710 mini_ranger_kms.cc:310] Created kms service
I20260809 16:21:55.853467 21710 mini_ranger_kms.cc:326] Created kudu user
I20260809 16:21:55.898524 21710 mini_ranger_kms.cc:343] Created rangerkms user
Aug 09 16:21:55 dist-test-slave-s11t krb5kdc[6109](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292513, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.21.51.148@KRBTEST.COM
I20260809 16:21:56.331653 21710 mini_ranger_kms.cc:384] Added ranger policy
I20260809 16:21:56.331774 21710 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786292417722232-21710-0/minicluster-data/ranger-kms:/tmp/dist-test-taskrNqUCn/build/release/bin/postgresql.jar:/tmp/dist-test-taskrNqUCn/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskrNqUCn/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260809 16:21:56.331794 21710 mini_ranger_kms.cc:240] Using host: 127.21.51.148
I20260809 16:21:56.335915 21710 mini_ranger_kms.cc:276] Ranger KMS PID: 6709
I20260809 16:21:56.336051 21710 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.21.51.148:37359
Aug 09, 2026 4:21:57 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 09, 2026 4:21:57 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 09, 2026 4:21:57 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786292417722232-21710-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 09, 2026 4:21:57 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786292417722232-21710-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 09, 2026 4:21:57 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786292417722232-21710-0/minicluster-data/ranger-kms/ews/webapp].
Aug 09, 2026 4:21:57 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 09 16:21:58 dist-test-slave-s11t krb5kdc[6109](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786292518, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.21.51.148@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:21:59.160454 21710 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.839s user 0.008s sys 0.273s
I20260809 16:21:59.160573 21710 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260809 16:21:59.160652 21710 mini_ranger_kms.cc:401] 127.21.51.148:37359/kms/v1/keys
Aug 09 16:21:59 dist-test-slave-s11t krb5kdc[6109](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292518, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.21.51.148@KRBTEST.COM for HTTP/127.21.51.148@KRBTEST.COM
Aug 09 16:21:59 dist-test-slave-s11t krb5kdc[6109](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292518, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.21.51.148@KRBTEST.COM for HTTP/127.21.51.148@KRBTEST.COM
Aug 09 16:22:01 dist-test-slave-s11t krb5kdc[6109](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292521, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.21.51.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.51.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.51.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.190.keytab.
Entry for principal kudu/127.21.51.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.190.keytab.
I20260809 16:22:01.917222 21710 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrNqUCn/build/release/bin/kudu
/tmp/dist-test-taskrNqUCn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786292417722232-21710-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786292417722232-21710-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786292417722232-21710-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786292417722232-21710-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.51.190:46779
--webserver_interface=127.21.51.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.190.keytab
--principal=kudu/127.21.51.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.51.148:37425
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.51.190:46779
--ranger_config_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786292417722232-21710-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.21.51.148:37359/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786292417722232-21710-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786292417722232-21710-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786292417722232-21710-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:22:02.002916 6789 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:22:02.003125 6789 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:22:02.003147 6789 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:22:02.004422 6789 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 16:22:02.004470 6789 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:22:02.004485 6789 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:22:02.004498 6789 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 16:22:02.004509 6789 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 16:22:02.006155 6789 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.51.148:37425
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786292417722232-21710-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786292417722232-21710-0/minicluster-data/master-0/wal
--ranger_kms_url=127.21.51.148:37359/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.21.51.190:46779
--ranger_config_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786292417722232-21710-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.190.keytab
--principal=kudu/127.21.51.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.51.190:46779
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786292417722232-21710-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.51.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6789
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786292417722232-21710-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:22:02.006412 6789 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:22:02.006668 6789 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:22:02.012147 6799 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:22:02.012148 6802 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:22:02.012189 6789 server_base.cc:1034] running on GCE node
W20260809 16:22:02.012189 6800 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:22:02.012603 6789 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:22:02.012928 6789 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:22:02.014129 6789 hybrid_clock.cc:648] HybridClock initialized: now 1786292522014114 us; error 44 us; skew 500 ppm
Aug 09 16:22:02 dist-test-slave-s11t krb5kdc[6109](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292522, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.51.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:22:02.020762 6789 init.cc:377] Logged in from keytab as kudu/127.21.51.190@KRBTEST.COM (short username kudu)
I20260809 16:22:02.021221 6789 webserver.cc:460] Webserver started at http://127.21.51.190:36469/ using document root <none> and password file <none>
I20260809 16:22:02.021490 6789 fs_manager.cc:359] Metadata directory not provided
I20260809 16:22:02.021559 6789 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:22:02.021718 6789 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 16:22:02 dist-test-slave-s11t krb5kdc[6109](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292522, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.51.190@KRBTEST.COM for HTTP/127.21.51.148@KRBTEST.COM
I20260809 16:22:02.078848 6789 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786292417722232-21710-0/minicluster-data/master-0/data/instance:
uuid: "b2a198ab748a408480a8aff11d6a11b4"
format_stamp: "Formatted at 2026-08-09 16:22:02 on dist-test-slave-s11t"
server_key: "ea7e99a70a9ee7a7a62f0d023dc13779"
server_key_iv: "2c1454595beacad2ec2900cb06dd1b99"
server_key_version: "kuduclusterkey@0"
I20260809 16:22:02.079317 6789 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786292417722232-21710-0/minicluster-data/master-0/wal/instance:
uuid: "b2a198ab748a408480a8aff11d6a11b4"
format_stamp: "Formatted at 2026-08-09 16:22:02 on dist-test-slave-s11t"
server_key: "ea7e99a70a9ee7a7a62f0d023dc13779"
server_key_iv: "2c1454595beacad2ec2900cb06dd1b99"
server_key_version: "kuduclusterkey@0"
I20260809 16:22:02.081444 6789 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.004s sys 0.000s
I20260809 16:22:02.095867 6811 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:22:02.096165 6789 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260809 16:22:02.096287 6789 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786292417722232-21710-0/minicluster-data/master-0/data,/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786292417722232-21710-0/minicluster-data/master-0/wal
uuid: "b2a198ab748a408480a8aff11d6a11b4"
format_stamp: "Formatted at 2026-08-09 16:22:02 on dist-test-slave-s11t"
server_key: "ea7e99a70a9ee7a7a62f0d023dc13779"
server_key_iv: "2c1454595beacad2ec2900cb06dd1b99"
server_key_version: "kuduclusterkey@0"
I20260809 16:22:02.096395 6789 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786292417722232-21710-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786292417722232-21710-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786292417722232-21710-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:22:02.111089 6789 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:22:02.113113 6789 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:22:02.113265 6789 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:22:02.118517 6789 rpc_server.cc:307] RPC server started. Bound to: 127.21.51.190:46779
I20260809 16:22:02.118566 6898 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.51.190:46779 every 8 connection(s)
I20260809 16:22:02.118940 6789 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786292417722232-21710-0/minicluster-data/master-0/data/info.pb
I20260809 16:22:02.119730 6899 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:22:02.122944 6899 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b2a198ab748a408480a8aff11d6a11b4: Bootstrap starting.
I20260809 16:22:02.123912 6899 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b2a198ab748a408480a8aff11d6a11b4: Neither blocks nor log segments found. Creating new log.
I20260809 16:22:02.124195 21710 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrNqUCn/build/release/bin/kudu as pid 6789
I20260809 16:22:02.124280 6899 log.cc:824] T 00000000000000000000000000000000 P b2a198ab748a408480a8aff11d6a11b4: Log is configured to *not* fsync() on all Append() calls
I20260809 16:22:02.124337 21710 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786292417722232-21710-0/minicluster-data/master-0/wal/instance
I20260809 16:22:02.124550 21710 external_mini_cluster.cc:1428] Setting key c054b38d20b4cd8d8c05272817eb1d53
I20260809 16:22:02.125237 6899 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b2a198ab748a408480a8aff11d6a11b4: No bootstrap required, opened a new log
I20260809 16:22:02.127959 6899 raft_consensus.cc:348] T 00000000000000000000000000000000 P b2a198ab748a408480a8aff11d6a11b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b2a198ab748a408480a8aff11d6a11b4" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 46779 } }
I20260809 16:22:02.128144 6899 raft_consensus.cc:374] T 00000000000000000000000000000000 P b2a198ab748a408480a8aff11d6a11b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:22:02.128181 6899 raft_consensus.cc:729] T 00000000000000000000000000000000 P b2a198ab748a408480a8aff11d6a11b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b2a198ab748a408480a8aff11d6a11b4, State: Initialized, Role: FOLLOWER
I20260809 16:22:02.128364 6899 consensus_queue.cc:260] T 00000000000000000000000000000000 P b2a198ab748a408480a8aff11d6a11b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b2a198ab748a408480a8aff11d6a11b4" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 46779 } }
I20260809 16:22:02.128445 6899 raft_consensus.cc:388] T 00000000000000000000000000000000 P b2a198ab748a408480a8aff11d6a11b4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 16:22:02.128474 6899 raft_consensus.cc:482] T 00000000000000000000000000000000 P b2a198ab748a408480a8aff11d6a11b4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 16:22:02.128540 6899 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b2a198ab748a408480a8aff11d6a11b4 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:22:02.129277 6899 raft_consensus.cc:504] T 00000000000000000000000000000000 P b2a198ab748a408480a8aff11d6a11b4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b2a198ab748a408480a8aff11d6a11b4" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 46779 } }
I20260809 16:22:02.129413 6899 leader_election.cc:302] T 00000000000000000000000000000000 P b2a198ab748a408480a8aff11d6a11b4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b2a198ab748a408480a8aff11d6a11b4; no voters:
I20260809 16:22:02.129582 6899 leader_election.cc:288] T 00000000000000000000000000000000 P b2a198ab748a408480a8aff11d6a11b4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 16:22:02.129611 6906 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b2a198ab748a408480a8aff11d6a11b4 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:22:02.129835 6906 raft_consensus.cc:686] T 00000000000000000000000000000000 P b2a198ab748a408480a8aff11d6a11b4 [term 1 LEADER]: Becoming Leader. State: Replica: b2a198ab748a408480a8aff11d6a11b4, State: Running, Role: LEADER
I20260809 16:22:02.129981 6906 consensus_queue.cc:237] T 00000000000000000000000000000000 P b2a198ab748a408480a8aff11d6a11b4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b2a198ab748a408480a8aff11d6a11b4" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 46779 } }
I20260809 16:22:02.129882 6899 sys_catalog.cc:564] T 00000000000000000000000000000000 P b2a198ab748a408480a8aff11d6a11b4 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 16:22:02.131908 6899 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786292417722232-21710-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260809 16:22:02.132005 6909 sys_catalog.cc:455] T 00000000000000000000000000000000 P b2a198ab748a408480a8aff11d6a11b4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b2a198ab748a408480a8aff11d6a11b4. Latest consensus state: current_term: 1 leader_uuid: "b2a198ab748a408480a8aff11d6a11b4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b2a198ab748a408480a8aff11d6a11b4" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 46779 } } }
I20260809 16:22:02.132052 6909 sys_catalog.cc:458] T 00000000000000000000000000000000 P b2a198ab748a408480a8aff11d6a11b4 [sys.catalog]: This master's current role is: LEADER
I20260809 16:22:02.131860 6907 sys_catalog.cc:455] T 00000000000000000000000000000000 P b2a198ab748a408480a8aff11d6a11b4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b2a198ab748a408480a8aff11d6a11b4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b2a198ab748a408480a8aff11d6a11b4" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 46779 } } }
I20260809 16:22:02.132334 6907 sys_catalog.cc:458] T 00000000000000000000000000000000 P b2a198ab748a408480a8aff11d6a11b4 [sys.catalog]: This master's current role is: LEADER
Aug 09 16:22:02 dist-test-slave-s11t krb5kdc[6109](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292521, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.51.190@KRBTEST.COM
I20260809 16:22:02.137068 6925 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 16:22:02.138074 6925 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 16:22:02.141773 6905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:22:02.125313 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45558 (local address 127.21.51.190:46779)
0809 16:22:02.125501 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0809 16:22:02.125509 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 16:22:02.125543 (+ 34us) server_negotiation.cc:407] Connection header received
0809 16:22:02.126562 (+ 1019us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:22:02.126571 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:22:02.126636 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:22:02.126794 (+ 158us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:22:02.127597 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:22:02.129063 (+ 1466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:22:02.130302 (+ 1239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:22:02.130984 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:22:02.131129 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:22:02.135324 (+ 4195us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:22:02.135371 (+ 47us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:22:02.135371 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:22:02.135480 (+ 109us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:22:02.138655 (+ 3175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:22:02.139085 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:22:02.139094 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:22:02.139095 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:22:02.139219 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:22:02.139459 (+ 240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:22:02.139460 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:22:02.139460 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:22:02.140106 (+ 646us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:22:02.140305 (+ 199us) server_negotiation.cc:1035] Waiting for connection context
0809 16:22:02.140378 (+ 73us) server_negotiation.cc:299] Negotiation successful
0809 16:22:02.141560 (+ 1182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":137,"thread_start_us":84,"threads_started":1}
I20260809 16:22:02.149209 6925 catalog_manager.cc:1347] Generated new cluster ID: e8e3015124d94ec897092120990eb2ad
I20260809 16:22:02.149266 6925 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 16:22:02.183359 6925 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 16:22:02.184235 6925 catalog_manager.cc:1504] Loading token signing keys...
I20260809 16:22:02.189191 6925 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b2a198ab748a408480a8aff11d6a11b4: Generated new TSK 0
I20260809 16:22:02.189455 6925 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.21.51.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.51.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.51.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.129.keytab.
Entry for principal kudu/127.21.51.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.129.keytab.
I20260809 16:22:02.903105 21710 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrNqUCn/build/release/bin/kudu
/tmp/dist-test-taskrNqUCn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786292417722232-21710-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786292417722232-21710-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786292417722232-21710-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786292417722232-21710-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.21.51.148:37359/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.51.129:0
--local_ip_for_outbound_sockets=127.21.51.129
--webserver_interface=127.21.51.129
--webserver_port=0
--tserver_master_addrs=127.21.51.190:46779
--keytab_file=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.129.keytab
--principal=kudu/127.21.51.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.51.148:37425
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786292417722232-21710-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786292417722232-21710-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786292417722232-21710-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:22:02.993894 6968 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:22:02.994107 6968 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:22:02.994131 6968 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:22:02.995347 6968 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:22:02.995391 6968 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:22:02.995424 6968 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.51.129
I20260809 16:22:02.996984 6968 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.51.148:37425
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786292417722232-21710-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786292417722232-21710-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.21.51.148:37359/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.129.keytab
--principal=kudu/127.21.51.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.51.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786292417722232-21710-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.51.129
--webserver_port=0
--tserver_master_addrs=127.21.51.190:46779
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6968
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.51.129
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786292417722232-21710-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:22:02.997373 6968 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:22:02.997655 6968 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:22:03.003364 6975 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:22:03.003496 6980 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:22:03.003646 6985 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:22:03.004076 6968 server_base.cc:1034] running on GCE node
I20260809 16:22:03.004292 6968 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:22:03.004556 6968 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:22:03.005730 6968 hybrid_clock.cc:648] HybridClock initialized: now 1786292523005702 us; error 41 us; skew 500 ppm
Aug 09 16:22:03 dist-test-slave-s11t krb5kdc[6109](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292523, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.51.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:22:03.008637 6968 init.cc:377] Logged in from keytab as kudu/127.21.51.129@KRBTEST.COM (short username kudu)
I20260809 16:22:03.009203 6968 webserver.cc:460] Webserver started at http://127.21.51.129:37161/ using document root <none> and password file <none>
I20260809 16:22:03.009481 6968 fs_manager.cc:359] Metadata directory not provided
I20260809 16:22:03.009557 6968 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:22:03.009756 6968 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 16:22:03 dist-test-slave-s11t krb5kdc[6109](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292523, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.51.129@KRBTEST.COM for HTTP/127.21.51.148@KRBTEST.COM
I20260809 16:22:03.024461 6968 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786292417722232-21710-0/minicluster-data/ts-0/data/instance:
uuid: "cfacc3aadc964583af4135ba9b1b662a"
format_stamp: "Formatted at 2026-08-09 16:22:03 on dist-test-slave-s11t"
server_key: "84558ac0c7da6038cc39be44c8dd1844"
server_key_iv: "51063686f32d8ded8b9af03fb1a4a53d"
server_key_version: "kuduclusterkey@0"
I20260809 16:22:03.025098 6968 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786292417722232-21710-0/minicluster-data/ts-0/wal/instance:
uuid: "cfacc3aadc964583af4135ba9b1b662a"
format_stamp: "Formatted at 2026-08-09 16:22:03 on dist-test-slave-s11t"
server_key: "84558ac0c7da6038cc39be44c8dd1844"
server_key_iv: "51063686f32d8ded8b9af03fb1a4a53d"
server_key_version: "kuduclusterkey@0"
I20260809 16:22:03.027415 6968 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20260809 16:22:03.041217 6996 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:22:03.041464 6968 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.002s sys 0.000s
I20260809 16:22:03.041572 6968 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786292417722232-21710-0/minicluster-data/ts-0/data,/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786292417722232-21710-0/minicluster-data/ts-0/wal
uuid: "cfacc3aadc964583af4135ba9b1b662a"
format_stamp: "Formatted at 2026-08-09 16:22:03 on dist-test-slave-s11t"
server_key: "84558ac0c7da6038cc39be44c8dd1844"
server_key_iv: "51063686f32d8ded8b9af03fb1a4a53d"
server_key_version: "kuduclusterkey@0"
I20260809 16:22:03.041694 6968 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786292417722232-21710-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786292417722232-21710-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786292417722232-21710-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:22:03.058147 6968 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:22:03.060894 6968 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:22:03.061081 6968 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:22:03.061367 6968 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:22:03.061770 6968 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:22:03.061878 6968 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:22:03.061997 6968 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:22:03.062084 6968 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:22:03.070247 6968 rpc_server.cc:307] RPC server started. Bound to: 127.21.51.129:33215
I20260809 16:22:03.070644 6968 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786292417722232-21710-0/minicluster-data/ts-0/data/info.pb
I20260809 16:22:03.077034 7175 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.51.129:33215 every 8 connection(s)
Aug 09 16:22:03 dist-test-slave-s11t krb5kdc[6109](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292523, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.51.129@KRBTEST.COM for kudu/127.21.51.190@KRBTEST.COM
I20260809 16:22:03.079588 21710 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrNqUCn/build/release/bin/kudu as pid 6968
I20260809 16:22:03.079701 21710 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786292417722232-21710-0/minicluster-data/ts-0/wal/instance
I20260809 16:22:03.079928 21710 external_mini_cluster.cc:1428] Setting key ae7fa0eaedf04a12e613946ee2f7326e
I20260809 16:22:03.081681 7180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:22:03.071460 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.51.129:53013 (local address 127.21.51.190:46779)
0809 16:22:03.071621 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0809 16:22:03.071623 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:22:03.071835 (+ 212us) server_negotiation.cc:407] Connection header received
0809 16:22:03.072423 (+ 588us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:22:03.072424 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:22:03.072439 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:22:03.072511 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:22:03.073420 (+ 909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:22:03.073943 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:22:03.074981 (+ 1038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:22:03.075204 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:22:03.075292 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:22:03.077939 (+ 2647us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:22:03.077941 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:22:03.077942 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:22:03.077950 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:22:03.079588 (+ 1638us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:22:03.080255 (+ 667us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:22:03.080257 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:22:03.080257 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:22:03.080326 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:22:03.080797 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:22:03.080798 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:22:03.080799 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:22:03.080974 (+ 175us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:22:03.081024 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0809 16:22:03.081410 (+ 386us) server_negotiation.cc:299] Negotiation successful
0809 16:22:03.081522 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":133,"thread_start_us":94,"threads_started":1}
I20260809 16:22:03.082312 7179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:22:03.071453 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:46779 (local address 127.21.51.129:53013)
0809 16:22:03.071687 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0809 16:22:03.071727 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0809 16:22:03.072299 (+ 572us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:22:03.072582 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:22:03.072586 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:22:03.072923 (+ 337us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:22:03.073311 (+ 388us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:22:03.073316 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:22:03.074043 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:22:03.074044 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:22:03.074868 (+ 824us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:22:03.074870 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:22:03.075306 (+ 436us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:22:03.075307 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:22:03.075448 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:22:03.076282 (+ 834us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:22:03.076291 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:22:03.077827 (+ 1536us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:22:03.079697 (+ 1870us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:22:03.079704 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:22:03.079707 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:22:03.080054 (+ 347us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:22:03.080398 (+ 344us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:22:03.080399 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:22:03.080399 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:22:03.080690 (+ 291us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:22:03.081077 (+ 387us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:22:03.081084 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:22:03.081251 (+ 167us) client_negotiation.cc:769] Sending connection context
0809 16:22:03.081816 (+ 565us) client_negotiation.cc:240] Negotiation successful
0809 16:22:03.082022 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":93,"threads_started":1}
I20260809 16:22:03.083093 7176 heartbeater.cc:347] Connected to a master server at 127.21.51.190:46779
I20260809 16:22:03.083215 7176 heartbeater.cc:464] Registering TS with master...
I20260809 16:22:03.083410 7176 heartbeater.cc:511] Master 127.21.51.190:46779 requested a full tablet report, sending...
I20260809 16:22:03.083956 6845 ts_manager.cc:194] Registered new tserver with Master: cfacc3aadc964583af4135ba9b1b662a (127.21.51.129:33215)
I20260809 16:22:03.084573 6845 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.51.129@KRBTEST.COM'} at 127.21.51.129:53013
WARNING: no policy specified for kudu/127.21.51.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.51.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.51.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.130.keytab.
Entry for principal kudu/127.21.51.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.130.keytab.
I20260809 16:22:03.115371 21710 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrNqUCn/build/release/bin/kudu
/tmp/dist-test-taskrNqUCn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786292417722232-21710-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786292417722232-21710-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786292417722232-21710-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786292417722232-21710-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.21.51.148:37359/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.51.130:0
--local_ip_for_outbound_sockets=127.21.51.130
--webserver_interface=127.21.51.130
--webserver_port=0
--tserver_master_addrs=127.21.51.190:46779
--keytab_file=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.130.keytab
--principal=kudu/127.21.51.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.51.148:37425
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786292417722232-21710-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786292417722232-21710-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786292417722232-21710-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:22:03.232980 7184 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:22:03.233217 7184 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:22:03.233265 7184 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:22:03.234463 7184 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:22:03.234536 7184 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:22:03.234582 7184 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.51.130
I20260809 16:22:03.236091 7184 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.51.148:37425
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786292417722232-21710-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786292417722232-21710-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.21.51.148:37359/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.130.keytab
--principal=kudu/127.21.51.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.51.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786292417722232-21710-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.51.130
--webserver_port=0
--tserver_master_addrs=127.21.51.190:46779
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7184
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.51.130
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786292417722232-21710-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:22:03.236364 7184 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:22:03.236603 7184 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:22:03.241803 7198 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:22:03.241848 7195 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:22:03.242182 7193 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:22:03.243003 7184 server_base.cc:1034] running on GCE node
I20260809 16:22:03.243362 7184 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:22:03.243672 7184 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:22:03.244858 7184 hybrid_clock.cc:648] HybridClock initialized: now 1786292523244839 us; error 32 us; skew 500 ppm
Aug 09 16:22:03 dist-test-slave-s11t krb5kdc[6109](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292523, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.51.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:22:03.247700 7184 init.cc:377] Logged in from keytab as kudu/127.21.51.130@KRBTEST.COM (short username kudu)
I20260809 16:22:03.248214 7184 webserver.cc:460] Webserver started at http://127.21.51.130:33793/ using document root <none> and password file <none>
I20260809 16:22:03.248464 7184 fs_manager.cc:359] Metadata directory not provided
I20260809 16:22:03.248534 7184 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:22:03.248720 7184 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 16:22:03 dist-test-slave-s11t krb5kdc[6109](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292523, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.51.130@KRBTEST.COM for HTTP/127.21.51.148@KRBTEST.COM
I20260809 16:22:03.261240 7184 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786292417722232-21710-0/minicluster-data/ts-1/data/instance:
uuid: "7f74f7c34f404fd2b8a074b249d86564"
format_stamp: "Formatted at 2026-08-09 16:22:03 on dist-test-slave-s11t"
server_key: "f57bc472fe591b4fa7adb963f680484e"
server_key_iv: "af818574a64b4e99159b411129dceee7"
server_key_version: "kuduclusterkey@0"
I20260809 16:22:03.261884 7184 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786292417722232-21710-0/minicluster-data/ts-1/wal/instance:
uuid: "7f74f7c34f404fd2b8a074b249d86564"
format_stamp: "Formatted at 2026-08-09 16:22:03 on dist-test-slave-s11t"
server_key: "f57bc472fe591b4fa7adb963f680484e"
server_key_iv: "af818574a64b4e99159b411129dceee7"
server_key_version: "kuduclusterkey@0"
I20260809 16:22:03.264773 7184 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
Aug 09 16:22:03 dist-test-slave-s11t krb5kdc[6109](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786292523, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.51.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:22:03.278265 7211 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:22:03.278515 7184 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 16:22:03.278630 7184 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786292417722232-21710-0/minicluster-data/ts-1/data,/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786292417722232-21710-0/minicluster-data/ts-1/wal
uuid: "7f74f7c34f404fd2b8a074b249d86564"
format_stamp: "Formatted at 2026-08-09 16:22:03 on dist-test-slave-s11t"
server_key: "f57bc472fe591b4fa7adb963f680484e"
server_key_iv: "af818574a64b4e99159b411129dceee7"
server_key_version: "kuduclusterkey@0"
I20260809 16:22:03.278734 7184 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786292417722232-21710-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786292417722232-21710-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786292417722232-21710-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:22:03.295995 7184 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:22:03.298578 7184 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:22:03.298796 7184 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:22:03.299109 7184 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:22:03.299547 7184 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:22:03.299641 7184 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:22:03.299724 7184 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:22:03.299796 7184 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:22:03.307814 7184 rpc_server.cc:307] RPC server started. Bound to: 127.21.51.130:41139
I20260809 16:22:03.308399 7184 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786292417722232-21710-0/minicluster-data/ts-1/data/info.pb
I20260809 16:22:03.309864 7376 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.51.130:41139 every 8 connection(s)
I20260809 16:22:03.316639 21710 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrNqUCn/build/release/bin/kudu as pid 7184
I20260809 16:22:03.316803 21710 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786292417722232-21710-0/minicluster-data/ts-1/wal/instance
I20260809 16:22:03.317055 21710 external_mini_cluster.cc:1428] Setting key df51ee58d47331658d879349dcaa6264
Aug 09 16:22:03 dist-test-slave-s11t krb5kdc[6109](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292523, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.51.130@KRBTEST.COM for kudu/127.21.51.190@KRBTEST.COM
I20260809 16:22:03.321879 7180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:22:03.310731 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.51.130:51955 (local address 127.21.51.190:46779)
0809 16:22:03.310791 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0809 16:22:03.310793 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:22:03.311088 (+ 295us) server_negotiation.cc:407] Connection header received
0809 16:22:03.311852 (+ 764us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:22:03.311854 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:22:03.311872 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:22:03.311905 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:22:03.312814 (+ 909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:22:03.313626 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:22:03.315180 (+ 1554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:22:03.315485 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:22:03.315531 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:22:03.318397 (+ 2866us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:22:03.318399 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:22:03.318399 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:22:03.318406 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:22:03.320176 (+ 1770us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:22:03.320678 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:22:03.320700 (+ 22us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:22:03.320701 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:22:03.320776 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:22:03.321078 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:22:03.321079 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:22:03.321080 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:22:03.321222 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:22:03.321268 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0809 16:22:03.321669 (+ 401us) server_negotiation.cc:299] Negotiation successful
0809 16:22:03.321780 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260809 16:22:03.322495 7380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:22:03.310797 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:46779 (local address 127.21.51.130:51955)
0809 16:22:03.310957 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0809 16:22:03.310998 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0809 16:22:03.311694 (+ 696us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:22:03.312100 (+ 406us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:22:03.312104 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:22:03.312366 (+ 262us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:22:03.312645 (+ 279us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:22:03.312651 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:22:03.313758 (+ 1107us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:22:03.313759 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:22:03.315070 (+ 1311us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:22:03.315073 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:22:03.315626 (+ 553us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:22:03.315627 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:22:03.315817 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:22:03.316496 (+ 679us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:22:03.316507 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:22:03.318284 (+ 1777us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:22:03.320277 (+ 1993us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:22:03.320284 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:22:03.320287 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:22:03.320578 (+ 291us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:22:03.320856 (+ 278us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:22:03.320856 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:22:03.320857 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:22:03.320992 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:22:03.321322 (+ 330us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:22:03.321328 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:22:03.321502 (+ 174us) client_negotiation.cc:769] Sending connection context
0809 16:22:03.322017 (+ 515us) client_negotiation.cc:240] Negotiation successful
0809 16:22:03.322189 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":93,"threads_started":1}
I20260809 16:22:03.323113 7377 heartbeater.cc:347] Connected to a master server at 127.21.51.190:46779
I20260809 16:22:03.323218 7377 heartbeater.cc:464] Registering TS with master...
I20260809 16:22:03.323414 7377 heartbeater.cc:511] Master 127.21.51.190:46779 requested a full tablet report, sending...
I20260809 16:22:03.323792 6845 ts_manager.cc:194] Registered new tserver with Master: 7f74f7c34f404fd2b8a074b249d86564 (127.21.51.130:41139)
I20260809 16:22:03.324354 6845 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.51.130@KRBTEST.COM'} at 127.21.51.130:51955
WARNING: no policy specified for kudu/127.21.51.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.51.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.51.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.131.keytab.
Entry for principal kudu/127.21.51.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.131.keytab.
I20260809 16:22:03.351876 21710 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrNqUCn/build/release/bin/kudu
/tmp/dist-test-taskrNqUCn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786292417722232-21710-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786292417722232-21710-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786292417722232-21710-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786292417722232-21710-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.21.51.148:37359/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.51.131:0
--local_ip_for_outbound_sockets=127.21.51.131
--webserver_interface=127.21.51.131
--webserver_port=0
--tserver_master_addrs=127.21.51.190:46779
--keytab_file=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.131.keytab
--principal=kudu/127.21.51.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.51.148:37425
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786292417722232-21710-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786292417722232-21710-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786292417722232-21710-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:22:03.476366 7385 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:22:03.476756 7385 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:22:03.476851 7385 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:22:03.478605 7385 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:22:03.478675 7385 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:22:03.478729 7385 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.51.131
I20260809 16:22:03.480955 7385 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.51.148:37425
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786292417722232-21710-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786292417722232-21710-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.21.51.148:37359/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.131.keytab
--principal=kudu/127.21.51.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.51.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786292417722232-21710-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.51.131
--webserver_port=0
--tserver_master_addrs=127.21.51.190:46779
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7385
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.51.131
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786292417722232-21710-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:22:03.481217 7385 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:22:03.481518 7385 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:22:03.488524 7395 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:22:03.488627 7392 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:22:03.489086 7385 server_base.cc:1034] running on GCE node
W20260809 16:22:03.489189 7397 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:22:03.489483 7385 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:22:03.489850 7385 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:22:03.491052 7385 hybrid_clock.cc:648] HybridClock initialized: now 1786292523491018 us; error 54 us; skew 500 ppm
Aug 09 16:22:03 dist-test-slave-s11t krb5kdc[6109](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292523, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.51.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:22:03.494815 7385 init.cc:377] Logged in from keytab as kudu/127.21.51.131@KRBTEST.COM (short username kudu)
I20260809 16:22:03.495360 7385 webserver.cc:460] Webserver started at http://127.21.51.131:45835/ using document root <none> and password file <none>
I20260809 16:22:03.495628 7385 fs_manager.cc:359] Metadata directory not provided
I20260809 16:22:03.495702 7385 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:22:03.495841 7385 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 16:22:03 dist-test-slave-s11t krb5kdc[6109](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292523, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.51.131@KRBTEST.COM for HTTP/127.21.51.148@KRBTEST.COM
I20260809 16:22:03.526341 7385 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786292417722232-21710-0/minicluster-data/ts-2/data/instance:
uuid: "0890b7adf04c424dac299041e01ae6fe"
format_stamp: "Formatted at 2026-08-09 16:22:03 on dist-test-slave-s11t"
server_key: "2babf11ad9e9ef24c73b488fc70eb4ad"
server_key_iv: "393893ae8f89edccd7d560c3581fadc3"
server_key_version: "kuduclusterkey@0"
I20260809 16:22:03.526968 7385 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786292417722232-21710-0/minicluster-data/ts-2/wal/instance:
uuid: "0890b7adf04c424dac299041e01ae6fe"
format_stamp: "Formatted at 2026-08-09 16:22:03 on dist-test-slave-s11t"
server_key: "2babf11ad9e9ef24c73b488fc70eb4ad"
server_key_iv: "393893ae8f89edccd7d560c3581fadc3"
server_key_version: "kuduclusterkey@0"
I20260809 16:22:03.537210 7385 fs_manager.cc:691] Time spent creating directory manager: real 0.010s user 0.002s sys 0.003s
I20260809 16:22:03.553637 7408 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:22:03.554008 7385 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260809 16:22:03.554159 7385 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786292417722232-21710-0/minicluster-data/ts-2/data,/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786292417722232-21710-0/minicluster-data/ts-2/wal
uuid: "0890b7adf04c424dac299041e01ae6fe"
format_stamp: "Formatted at 2026-08-09 16:22:03 on dist-test-slave-s11t"
server_key: "2babf11ad9e9ef24c73b488fc70eb4ad"
server_key_iv: "393893ae8f89edccd7d560c3581fadc3"
server_key_version: "kuduclusterkey@0"
I20260809 16:22:03.554308 7385 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786292417722232-21710-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786292417722232-21710-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786292417722232-21710-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:22:03.570700 7385 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:22:03.572966 7385 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:22:03.573144 7385 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:22:03.573400 7385 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:22:03.573732 7385 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:22:03.573791 7385 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:22:03.573838 7385 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:22:03.573869 7385 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:22:03.581449 7385 rpc_server.cc:307] RPC server started. Bound to: 127.21.51.131:34699
I20260809 16:22:03.592942 7583 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.51.131:34699 every 8 connection(s)
I20260809 16:22:03.593356 7385 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786292417722232-21710-0/minicluster-data/ts-2/data/info.pb
I20260809 16:22:03.596065 21710 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrNqUCn/build/release/bin/kudu as pid 7385
I20260809 16:22:03.596179 21710 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786292417722232-21710-0/minicluster-data/ts-2/wal/instance
I20260809 16:22:03.596390 21710 external_mini_cluster.cc:1428] Setting key 0181db30f3c3c50eed1162a5ed249e87
Aug 09 16:22:03 dist-test-slave-s11t krb5kdc[6109](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292523, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.51.131@KRBTEST.COM for kudu/127.21.51.190@KRBTEST.COM
I20260809 16:22:03.607435 7180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:22:03.594232 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.51.131:54825 (local address 127.21.51.190:46779)
0809 16:22:03.594298 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0809 16:22:03.594300 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:22:03.594557 (+ 257us) server_negotiation.cc:407] Connection header received
0809 16:22:03.595340 (+ 783us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:22:03.595342 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:22:03.595358 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:22:03.595394 (+ 36us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:22:03.596488 (+ 1094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:22:03.597451 (+ 963us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:22:03.599693 (+ 2242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:22:03.600079 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:22:03.600133 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:22:03.603646 (+ 3513us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:22:03.603648 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:22:03.603648 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:22:03.603657 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:22:03.605543 (+ 1886us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:22:03.606099 (+ 556us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:22:03.606100 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:22:03.606100 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:22:03.606172 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:22:03.606604 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:22:03.606605 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:22:03.606605 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:22:03.606761 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:22:03.606848 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0809 16:22:03.607222 (+ 374us) server_negotiation.cc:299] Negotiation successful
0809 16:22:03.607322 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260809 16:22:03.607599 7588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:22:03.594198 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:46779 (local address 127.21.51.131:54825)
0809 16:22:03.594403 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0809 16:22:03.594456 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0809 16:22:03.595198 (+ 742us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:22:03.595601 (+ 403us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:22:03.595606 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:22:03.595937 (+ 331us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:22:03.596333 (+ 396us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:22:03.596340 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:22:03.598029 (+ 1689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:22:03.598030 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:22:03.599573 (+ 1543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:22:03.599576 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:22:03.600857 (+ 1281us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:22:03.600858 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:22:03.601039 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:22:03.601694 (+ 655us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:22:03.601706 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:22:03.603530 (+ 1824us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:22:03.605644 (+ 2114us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:22:03.605651 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:22:03.605653 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:22:03.606004 (+ 351us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:22:03.606375 (+ 371us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:22:03.606376 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:22:03.606377 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:22:03.606526 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:22:03.606864 (+ 338us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:22:03.606871 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:22:03.607042 (+ 171us) client_negotiation.cc:769] Sending connection context
0809 16:22:03.607170 (+ 128us) client_negotiation.cc:240] Negotiation successful
0809 16:22:03.607354 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":99,"threads_started":1}
I20260809 16:22:03.608168 7585 heartbeater.cc:347] Connected to a master server at 127.21.51.190:46779
I20260809 16:22:03.608287 7585 heartbeater.cc:464] Registering TS with master...
I20260809 16:22:03.608500 7585 heartbeater.cc:511] Master 127.21.51.190:46779 requested a full tablet report, sending...
I20260809 16:22:03.609010 6845 ts_manager.cc:194] Registered new tserver with Master: 0890b7adf04c424dac299041e01ae6fe (127.21.51.131:34699)
I20260809 16:22:03.609546 6845 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.51.131@KRBTEST.COM'} at 127.21.51.131:54825
I20260809 16:22:03.613106 21710 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 16:22:03.624152 7180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:22:03.614935 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45562 (local address 127.21.51.190:46779)
0809 16:22:03.614978 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0809 16:22:03.614980 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:22:03.615120 (+ 140us) server_negotiation.cc:407] Connection header received
0809 16:22:03.615197 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:22:03.615198 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:22:03.615211 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:22:03.615285 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:22:03.616057 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:22:03.616928 (+ 871us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:22:03.618317 (+ 1389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:22:03.618634 (+ 317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:22:03.618684 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:22:03.620517 (+ 1833us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:22:03.620518 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:22:03.620519 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:22:03.620527 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:22:03.622676 (+ 2149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:22:03.623162 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:22:03.623164 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:22:03.623164 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:22:03.623227 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:22:03.623471 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:22:03.623472 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:22:03.623472 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:22:03.623613 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:22:03.623696 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0809 16:22:03.623951 (+ 255us) server_negotiation.cc:299] Negotiation successful
0809 16:22:03.624043 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260809 16:22:03.626937 6845 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45562:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 16:22:03.629477 6845 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 16:22:03.638880 7603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:22:03.632675 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.130:41139 (local address 127.0.0.1:43210)
0809 16:22:03.633776 (+ 1101us) negotiation.cc:107] Waiting for socket to connect
0809 16:22:03.633785 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 16:22:03.633863 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:22:03.634143 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:22:03.634144 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:22:03.634146 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:22:03.634194 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:22:03.634196 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:22:03.635846 (+ 1650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:22:03.635847 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:22:03.637118 (+ 1271us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:22:03.637121 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:22:03.638593 (+ 1472us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:22:03.638594 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:22:03.638662 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:22:03.638669 (+ 7us) client_negotiation.cc:769] Sending connection context
0809 16:22:03.638726 (+ 57us) client_negotiation.cc:240] Negotiation successful
0809 16:22:03.638786 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1085,"mutex_wait_us":3,"thread_start_us":112,"threads_started":1}
I20260809 16:22:03.639086 7602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:22:03.632553 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37380 (local address 127.21.51.131:34699)
0809 16:22:03.633074 (+ 521us) server_negotiation.cc:206] Beginning negotiation
0809 16:22:03.633079 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 16:22:03.633604 (+ 525us) server_negotiation.cc:407] Connection header received
0809 16:22:03.633789 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:22:03.633792 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:22:03.633821 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:22:03.633920 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:22:03.634213 (+ 293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:22:03.635498 (+ 1285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:22:03.637275 (+ 1777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:22:03.638160 (+ 885us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:22:03.638243 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:22:03.638504 (+ 261us) server_negotiation.cc:1035] Waiting for connection context
0809 16:22:03.638953 (+ 449us) server_negotiation.cc:299] Negotiation successful
0809 16:22:03.639000 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":475,"thread_start_us":130,"threads_started":1}
I20260809 16:22:03.639348 7606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:22:03.632841 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43210 (local address 127.21.51.130:41139)
0809 16:22:03.633153 (+ 312us) server_negotiation.cc:206] Beginning negotiation
0809 16:22:03.633157 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 16:22:03.633837 (+ 680us) server_negotiation.cc:407] Connection header received
0809 16:22:03.633950 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:22:03.633953 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:22:03.633969 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:22:03.634020 (+ 51us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:22:03.634283 (+ 263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:22:03.635741 (+ 1458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:22:03.637286 (+ 1545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:22:03.638401 (+ 1115us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:22:03.638455 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:22:03.638525 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0809 16:22:03.639206 (+ 681us) server_negotiation.cc:299] Negotiation successful
0809 16:22:03.639263 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":281,"thread_start_us":96,"threads_started":1}
I20260809 16:22:03.638906 7612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:22:03.632424 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.131:34699 (local address 127.0.0.1:37380)
0809 16:22:03.633464 (+ 1040us) negotiation.cc:107] Waiting for socket to connect
0809 16:22:03.633505 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0809 16:22:03.633650 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:22:03.634029 (+ 379us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:22:03.634033 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:22:03.634036 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:22:03.634113 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:22:03.634115 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:22:03.635611 (+ 1496us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:22:03.635613 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:22:03.637142 (+ 1529us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:22:03.637145 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:22:03.638548 (+ 1403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:22:03.638549 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:22:03.638643 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:22:03.638660 (+ 17us) client_negotiation.cc:769] Sending connection context
0809 16:22:03.638741 (+ 81us) client_negotiation.cc:240] Negotiation successful
0809 16:22:03.638793 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1009,"mutex_wait_us":1,"thread_start_us":96,"threads_started":1}
I20260809 16:22:03.640394 7283 tablet_service.cc:1467] Processing CreateTablet for tablet e7694195883841cb86e1864f532ca596 (DEFAULT_TABLE table=test-table [id=c4ab85bfc5e048ca9aec5eb689fe041b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:22:03.640800 7283 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e7694195883841cb86e1864f532ca596. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:22:03.640625 7480 tablet_service.cc:1467] Processing CreateTablet for tablet e7694195883841cb86e1864f532ca596 (DEFAULT_TABLE table=test-table [id=c4ab85bfc5e048ca9aec5eb689fe041b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:22:03.641077 7480 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e7694195883841cb86e1864f532ca596. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:22:03.645231 7618 tablet_bootstrap.cc:492] T e7694195883841cb86e1864f532ca596 P 0890b7adf04c424dac299041e01ae6fe: Bootstrap starting.
I20260809 16:22:03.646591 7618 tablet_bootstrap.cc:654] T e7694195883841cb86e1864f532ca596 P 0890b7adf04c424dac299041e01ae6fe: Neither blocks nor log segments found. Creating new log.
I20260809 16:22:03.647058 7618 log.cc:824] T e7694195883841cb86e1864f532ca596 P 0890b7adf04c424dac299041e01ae6fe: Log is configured to *not* fsync() on all Append() calls
I20260809 16:22:03.648125 7618 tablet_bootstrap.cc:492] T e7694195883841cb86e1864f532ca596 P 0890b7adf04c424dac299041e01ae6fe: No bootstrap required, opened a new log
I20260809 16:22:03.648209 7618 ts_tablet_manager.cc:1397] T e7694195883841cb86e1864f532ca596 P 0890b7adf04c424dac299041e01ae6fe: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260809 16:22:03.649494 7603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:22:03.632889 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.129:33215 (local address 127.0.0.1:35820)
0809 16:22:03.644930 (+ 12041us) negotiation.cc:107] Waiting for socket to connect
0809 16:22:03.644942 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 16:22:03.644995 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:22:03.645305 (+ 310us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:22:03.645306 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:22:03.645308 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:22:03.645347 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:22:03.645348 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:22:03.646659 (+ 1311us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:22:03.646660 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:22:03.648153 (+ 1493us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:22:03.648156 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:22:03.649235 (+ 1079us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:22:03.649236 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:22:03.649314 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:22:03.649318 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 16:22:03.649375 (+ 57us) client_negotiation.cc:240] Negotiation successful
0809 16:22:03.649419 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12029,"thread_start_us":60,"threads_started":1}
I20260809 16:22:03.649791 7614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:22:03.633172 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35820 (local address 127.21.51.129:33215)
0809 16:22:03.633329 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0809 16:22:03.633334 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 16:22:03.645066 (+ 11732us) server_negotiation.cc:407] Connection header received
0809 16:22:03.645110 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:22:03.645113 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:22:03.645131 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:22:03.645207 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:22:03.645432 (+ 225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:22:03.646556 (+ 1124us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:22:03.648258 (+ 1702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:22:03.649060 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:22:03.649138 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:22:03.649197 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0809 16:22:03.649669 (+ 472us) server_negotiation.cc:299] Negotiation successful
0809 16:22:03.649718 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":121,"thread_start_us":73,"threads_started":1}
I20260809 16:22:03.650425 7072 tablet_service.cc:1467] Processing CreateTablet for tablet e7694195883841cb86e1864f532ca596 (DEFAULT_TABLE table=test-table [id=c4ab85bfc5e048ca9aec5eb689fe041b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:22:03.650439 7618 raft_consensus.cc:348] T e7694195883841cb86e1864f532ca596 P 0890b7adf04c424dac299041e01ae6fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0890b7adf04c424dac299041e01ae6fe" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 34699 } } peers { permanent_uuid: "7f74f7c34f404fd2b8a074b249d86564" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 41139 } } peers { permanent_uuid: "cfacc3aadc964583af4135ba9b1b662a" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 33215 } }
I20260809 16:22:03.650619 7618 raft_consensus.cc:374] T e7694195883841cb86e1864f532ca596 P 0890b7adf04c424dac299041e01ae6fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:22:03.650658 7618 raft_consensus.cc:729] T e7694195883841cb86e1864f532ca596 P 0890b7adf04c424dac299041e01ae6fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0890b7adf04c424dac299041e01ae6fe, State: Initialized, Role: FOLLOWER
I20260809 16:22:03.650770 7072 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e7694195883841cb86e1864f532ca596. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:22:03.651223 7618 consensus_queue.cc:260] T e7694195883841cb86e1864f532ca596 P 0890b7adf04c424dac299041e01ae6fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0890b7adf04c424dac299041e01ae6fe" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 34699 } } peers { permanent_uuid: "7f74f7c34f404fd2b8a074b249d86564" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 41139 } } peers { permanent_uuid: "cfacc3aadc964583af4135ba9b1b662a" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 33215 } }
I20260809 16:22:03.651623 7618 ts_tablet_manager.cc:1428] T e7694195883841cb86e1864f532ca596 P 0890b7adf04c424dac299041e01ae6fe: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 16:22:03.651834 7585 heartbeater.cc:503] Master 127.21.51.190:46779 was elected leader, sending a full tablet report...
I20260809 16:22:03.654397 7621 tablet_bootstrap.cc:492] T e7694195883841cb86e1864f532ca596 P cfacc3aadc964583af4135ba9b1b662a: Bootstrap starting.
I20260809 16:22:03.654984 7615 tablet_bootstrap.cc:492] T e7694195883841cb86e1864f532ca596 P 7f74f7c34f404fd2b8a074b249d86564: Bootstrap starting.
I20260809 16:22:03.655239 7621 tablet_bootstrap.cc:654] T e7694195883841cb86e1864f532ca596 P cfacc3aadc964583af4135ba9b1b662a: Neither blocks nor log segments found. Creating new log.
I20260809 16:22:03.655604 7621 log.cc:824] T e7694195883841cb86e1864f532ca596 P cfacc3aadc964583af4135ba9b1b662a: Log is configured to *not* fsync() on all Append() calls
I20260809 16:22:03.656288 7615 tablet_bootstrap.cc:654] T e7694195883841cb86e1864f532ca596 P 7f74f7c34f404fd2b8a074b249d86564: Neither blocks nor log segments found. Creating new log.
I20260809 16:22:03.656738 7615 log.cc:824] T e7694195883841cb86e1864f532ca596 P 7f74f7c34f404fd2b8a074b249d86564: Log is configured to *not* fsync() on all Append() calls
I20260809 16:22:03.657589 7615 tablet_bootstrap.cc:492] T e7694195883841cb86e1864f532ca596 P 7f74f7c34f404fd2b8a074b249d86564: No bootstrap required, opened a new log
I20260809 16:22:03.657712 7615 ts_tablet_manager.cc:1397] T e7694195883841cb86e1864f532ca596 P 7f74f7c34f404fd2b8a074b249d86564: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.001s
I20260809 16:22:03.660298 7615 raft_consensus.cc:348] T e7694195883841cb86e1864f532ca596 P 7f74f7c34f404fd2b8a074b249d86564 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0890b7adf04c424dac299041e01ae6fe" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 34699 } } peers { permanent_uuid: "7f74f7c34f404fd2b8a074b249d86564" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 41139 } } peers { permanent_uuid: "cfacc3aadc964583af4135ba9b1b662a" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 33215 } }
I20260809 16:22:03.660508 7615 raft_consensus.cc:374] T e7694195883841cb86e1864f532ca596 P 7f74f7c34f404fd2b8a074b249d86564 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:22:03.660641 7615 raft_consensus.cc:729] T e7694195883841cb86e1864f532ca596 P 7f74f7c34f404fd2b8a074b249d86564 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7f74f7c34f404fd2b8a074b249d86564, State: Initialized, Role: FOLLOWER
I20260809 16:22:03.660862 7615 consensus_queue.cc:260] T e7694195883841cb86e1864f532ca596 P 7f74f7c34f404fd2b8a074b249d86564 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0890b7adf04c424dac299041e01ae6fe" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 34699 } } peers { permanent_uuid: "7f74f7c34f404fd2b8a074b249d86564" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 41139 } } peers { permanent_uuid: "cfacc3aadc964583af4135ba9b1b662a" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 33215 } }
I20260809 16:22:03.661211 7615 ts_tablet_manager.cc:1428] T e7694195883841cb86e1864f532ca596 P 7f74f7c34f404fd2b8a074b249d86564: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260809 16:22:03.661382 7621 tablet_bootstrap.cc:492] T e7694195883841cb86e1864f532ca596 P cfacc3aadc964583af4135ba9b1b662a: No bootstrap required, opened a new log
I20260809 16:22:03.661389 7377 heartbeater.cc:503] Master 127.21.51.190:46779 was elected leader, sending a full tablet report...
I20260809 16:22:03.661540 7621 ts_tablet_manager.cc:1397] T e7694195883841cb86e1864f532ca596 P cfacc3aadc964583af4135ba9b1b662a: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.000s
I20260809 16:22:03.664467 7621 raft_consensus.cc:348] T e7694195883841cb86e1864f532ca596 P cfacc3aadc964583af4135ba9b1b662a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0890b7adf04c424dac299041e01ae6fe" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 34699 } } peers { permanent_uuid: "7f74f7c34f404fd2b8a074b249d86564" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 41139 } } peers { permanent_uuid: "cfacc3aadc964583af4135ba9b1b662a" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 33215 } }
I20260809 16:22:03.664717 7621 raft_consensus.cc:374] T e7694195883841cb86e1864f532ca596 P cfacc3aadc964583af4135ba9b1b662a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:22:03.664810 7621 raft_consensus.cc:729] T e7694195883841cb86e1864f532ca596 P cfacc3aadc964583af4135ba9b1b662a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cfacc3aadc964583af4135ba9b1b662a, State: Initialized, Role: FOLLOWER
I20260809 16:22:03.664999 7621 consensus_queue.cc:260] T e7694195883841cb86e1864f532ca596 P cfacc3aadc964583af4135ba9b1b662a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0890b7adf04c424dac299041e01ae6fe" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 34699 } } peers { permanent_uuid: "7f74f7c34f404fd2b8a074b249d86564" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 41139 } } peers { permanent_uuid: "cfacc3aadc964583af4135ba9b1b662a" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 33215 } }
I20260809 16:22:03.665334 7621 ts_tablet_manager.cc:1428] T e7694195883841cb86e1864f532ca596 P cfacc3aadc964583af4135ba9b1b662a: Time spent starting tablet: real 0.004s user 0.000s sys 0.003s
I20260809 16:22:03.665501 7176 heartbeater.cc:503] Master 127.21.51.190:46779 was elected leader, sending a full tablet report...
Aug 09 16:22:03 dist-test-slave-s11t krb5kdc[6109](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292523, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.51.190@KRBTEST.COM for HTTP/127.21.51.148@KRBTEST.COM
I20260809 16:22:03.781602 7630 raft_consensus.cc:482] T e7694195883841cb86e1864f532ca596 P cfacc3aadc964583af4135ba9b1b662a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 16:22:03.781775 7630 raft_consensus.cc:504] T e7694195883841cb86e1864f532ca596 P cfacc3aadc964583af4135ba9b1b662a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0890b7adf04c424dac299041e01ae6fe" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 34699 } } peers { permanent_uuid: "7f74f7c34f404fd2b8a074b249d86564" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 41139 } } peers { permanent_uuid: "cfacc3aadc964583af4135ba9b1b662a" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 33215 } }
I20260809 16:22:03.782219 7630 leader_election.cc:288] T e7694195883841cb86e1864f532ca596 P cfacc3aadc964583af4135ba9b1b662a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0890b7adf04c424dac299041e01ae6fe (127.21.51.131:34699), 7f74f7c34f404fd2b8a074b249d86564 (127.21.51.130:41139)
I20260809 16:22:03.789860 7602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:22:03.783134 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.51.129:51231 (local address 127.21.51.131:34699)
0809 16:22:03.783179 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0809 16:22:03.783181 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:22:03.785200 (+ 2019us) server_negotiation.cc:407] Connection header received
0809 16:22:03.785317 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:22:03.785318 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:22:03.785330 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:22:03.785371 (+ 41us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:22:03.785621 (+ 250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:22:03.786447 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:22:03.788802 (+ 2355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:22:03.789455 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:22:03.789509 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:22:03.789519 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0809 16:22:03.789745 (+ 226us) server_negotiation.cc:299] Negotiation successful
0809 16:22:03.789784 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260809 16:22:03.790352 7631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:22:03.782519 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.131:34699 (local address 127.21.51.129:51231)
0809 16:22:03.785117 (+ 2598us) negotiation.cc:107] Waiting for socket to connect
0809 16:22:03.785130 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 16:22:03.785237 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:22:03.785450 (+ 213us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:22:03.785451 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:22:03.785453 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:22:03.785541 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:22:03.785543 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:22:03.787303 (+ 1760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:22:03.787304 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:22:03.788672 (+ 1368us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:22:03.788675 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:22:03.789583 (+ 908us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:22:03.789584 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:22:03.789659 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:22:03.789663 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 16:22:03.790212 (+ 549us) client_negotiation.cc:240] Negotiation successful
0809 16:22:03.790262 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2577,"thread_start_us":136,"threads_started":1}
I20260809 16:22:03.791646 7513 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e7694195883841cb86e1864f532ca596" candidate_uuid: "cfacc3aadc964583af4135ba9b1b662a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0890b7adf04c424dac299041e01ae6fe" is_pre_election: true
I20260809 16:22:03.791847 7513 raft_consensus.cc:2454] T e7694195883841cb86e1864f532ca596 P 0890b7adf04c424dac299041e01ae6fe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cfacc3aadc964583af4135ba9b1b662a in term 0.
I20260809 16:22:03.792188 6999 leader_election.cc:302] T e7694195883841cb86e1864f532ca596 P cfacc3aadc964583af4135ba9b1b662a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0890b7adf04c424dac299041e01ae6fe, cfacc3aadc964583af4135ba9b1b662a; no voters:
I20260809 16:22:03.792320 7630 raft_consensus.cc:2781] T e7694195883841cb86e1864f532ca596 P cfacc3aadc964583af4135ba9b1b662a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 16:22:03.792371 7630 raft_consensus.cc:482] T e7694195883841cb86e1864f532ca596 P cfacc3aadc964583af4135ba9b1b662a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 16:22:03.792394 7630 raft_consensus.cc:3037] T e7694195883841cb86e1864f532ca596 P cfacc3aadc964583af4135ba9b1b662a [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:22:03.793339 7630 raft_consensus.cc:504] T e7694195883841cb86e1864f532ca596 P cfacc3aadc964583af4135ba9b1b662a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0890b7adf04c424dac299041e01ae6fe" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 34699 } } peers { permanent_uuid: "7f74f7c34f404fd2b8a074b249d86564" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 41139 } } peers { permanent_uuid: "cfacc3aadc964583af4135ba9b1b662a" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 33215 } }
I20260809 16:22:03.793480 7630 leader_election.cc:288] T e7694195883841cb86e1864f532ca596 P cfacc3aadc964583af4135ba9b1b662a [CANDIDATE]: Term 1 election: Requested vote from peers 0890b7adf04c424dac299041e01ae6fe (127.21.51.131:34699), 7f74f7c34f404fd2b8a074b249d86564 (127.21.51.130:41139)
I20260809 16:22:03.793802 7513 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e7694195883841cb86e1864f532ca596" candidate_uuid: "cfacc3aadc964583af4135ba9b1b662a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0890b7adf04c424dac299041e01ae6fe"
I20260809 16:22:03.793884 7513 raft_consensus.cc:3037] T e7694195883841cb86e1864f532ca596 P 0890b7adf04c424dac299041e01ae6fe [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:22:03.794872 7513 raft_consensus.cc:2454] T e7694195883841cb86e1864f532ca596 P 0890b7adf04c424dac299041e01ae6fe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cfacc3aadc964583af4135ba9b1b662a in term 1.
I20260809 16:22:03.795091 6999 leader_election.cc:302] T e7694195883841cb86e1864f532ca596 P cfacc3aadc964583af4135ba9b1b662a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0890b7adf04c424dac299041e01ae6fe, cfacc3aadc964583af4135ba9b1b662a; no voters:
I20260809 16:22:03.795204 7630 raft_consensus.cc:2781] T e7694195883841cb86e1864f532ca596 P cfacc3aadc964583af4135ba9b1b662a [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:22:03.795367 7630 raft_consensus.cc:686] T e7694195883841cb86e1864f532ca596 P cfacc3aadc964583af4135ba9b1b662a [term 1 LEADER]: Becoming Leader. State: Replica: cfacc3aadc964583af4135ba9b1b662a, State: Running, Role: LEADER
I20260809 16:22:03.795465 7630 consensus_queue.cc:237] T e7694195883841cb86e1864f532ca596 P cfacc3aadc964583af4135ba9b1b662a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0890b7adf04c424dac299041e01ae6fe" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 34699 } } peers { permanent_uuid: "7f74f7c34f404fd2b8a074b249d86564" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 41139 } } peers { permanent_uuid: "cfacc3aadc964583af4135ba9b1b662a" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 33215 } }
I20260809 16:22:03.796360 6845 catalog_manager.cc:5495] T e7694195883841cb86e1864f532ca596 P cfacc3aadc964583af4135ba9b1b662a reported cstate change: term changed from 0 to 1, leader changed from <none> to cfacc3aadc964583af4135ba9b1b662a (127.21.51.129). New cstate: current_term: 1 leader_uuid: "cfacc3aadc964583af4135ba9b1b662a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0890b7adf04c424dac299041e01ae6fe" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 34699 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7f74f7c34f404fd2b8a074b249d86564" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 41139 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cfacc3aadc964583af4135ba9b1b662a" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 33215 } health_report { overall_health: HEALTHY } } }
I20260809 16:22:03.798955 7606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:22:03.782854 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.51.129:32785 (local address 127.21.51.130:41139)
0809 16:22:03.782916 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0809 16:22:03.782918 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:22:03.794191 (+ 11273us) server_negotiation.cc:407] Connection header received
0809 16:22:03.794298 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:22:03.794299 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:22:03.794314 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:22:03.794345 (+ 31us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:22:03.794557 (+ 212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:22:03.795282 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:22:03.797987 (+ 2705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:22:03.798546 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:22:03.798603 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:22:03.798613 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0809 16:22:03.798834 (+ 221us) server_negotiation.cc:299] Negotiation successful
0809 16:22:03.798887 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260809 16:22:03.799396 7633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:22:03.782794 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.130:41139 (local address 127.21.51.129:32785)
0809 16:22:03.794112 (+ 11318us) negotiation.cc:107] Waiting for socket to connect
0809 16:22:03.794125 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 16:22:03.794221 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:22:03.794414 (+ 193us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:22:03.794415 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:22:03.794417 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:22:03.794481 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:22:03.794482 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:22:03.795404 (+ 922us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:22:03.795405 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:22:03.797877 (+ 2472us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:22:03.797880 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:22:03.798674 (+ 794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:22:03.798674 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:22:03.798753 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:22:03.798757 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 16:22:03.799195 (+ 438us) client_negotiation.cc:240] Negotiation successful
0809 16:22:03.799238 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11296,"thread_start_us":77,"threads_started":1}
I20260809 16:22:03.799882 7308 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e7694195883841cb86e1864f532ca596" candidate_uuid: "cfacc3aadc964583af4135ba9b1b662a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7f74f7c34f404fd2b8a074b249d86564" is_pre_election: true
I20260809 16:22:03.800043 7308 raft_consensus.cc:2454] T e7694195883841cb86e1864f532ca596 P 7f74f7c34f404fd2b8a074b249d86564 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cfacc3aadc964583af4135ba9b1b662a in term 0.
I20260809 16:22:03.800179 7301 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e7694195883841cb86e1864f532ca596" candidate_uuid: "cfacc3aadc964583af4135ba9b1b662a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7f74f7c34f404fd2b8a074b249d86564"
I20260809 16:22:03.800243 7301 raft_consensus.cc:3037] T e7694195883841cb86e1864f532ca596 P 7f74f7c34f404fd2b8a074b249d86564 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:22:03.801162 7301 raft_consensus.cc:2454] T e7694195883841cb86e1864f532ca596 P 7f74f7c34f404fd2b8a074b249d86564 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cfacc3aadc964583af4135ba9b1b662a in term 1.
W20260809 16:22:03.810930 7379 tablet.cc:2369] T e7694195883841cb86e1864f532ca596 P 7f74f7c34f404fd2b8a074b249d86564: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 16:22:03.827526 7614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:22:03.823642 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35822 (local address 127.21.51.129:33215)
0809 16:22:03.823703 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0809 16:22:03.823705 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:22:03.823717 (+ 12us) server_negotiation.cc:407] Connection header received
0809 16:22:03.823762 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:22:03.823763 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:22:03.823789 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:22:03.823885 (+ 96us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 16:22:03.824082 (+ 197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:22:03.824961 (+ 879us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:22:03.826228 (+ 1267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:22:03.826583 (+ 355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:22:03.826653 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:22:03.826880 (+ 227us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 16:22:03.827023 (+ 143us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 16:22:03.827103 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0809 16:22:03.827383 (+ 280us) server_negotiation.cc:299] Negotiation successful
0809 16:22:03.827408 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
W20260809 16:22:03.827859 7177 tablet.cc:2369] T e7694195883841cb86e1864f532ca596 P cfacc3aadc964583af4135ba9b1b662a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 16:22:03.830184 7301 raft_consensus.cc:1261] T e7694195883841cb86e1864f532ca596 P 7f74f7c34f404fd2b8a074b249d86564 [term 1 FOLLOWER]: Refusing update from remote peer cfacc3aadc964583af4135ba9b1b662a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 16:22:03.830703 7634 consensus_queue.cc:1035] T e7694195883841cb86e1864f532ca596 P cfacc3aadc964583af4135ba9b1b662a [LEADER]: Connected to new peer: Peer: permanent_uuid: "7f74f7c34f404fd2b8a074b249d86564" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 41139 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 16:22:03.833129 7513 raft_consensus.cc:1261] T e7694195883841cb86e1864f532ca596 P 0890b7adf04c424dac299041e01ae6fe [term 1 FOLLOWER]: Refusing update from remote peer cfacc3aadc964583af4135ba9b1b662a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 16:22:03.833537 7634 consensus_queue.cc:1035] T e7694195883841cb86e1864f532ca596 P cfacc3aadc964583af4135ba9b1b662a [LEADER]: Connected to new peer: Peer: permanent_uuid: "0890b7adf04c424dac299041e01ae6fe" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 34699 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 16:22:03.837939 7638 mvcc.cc:205] Tried to move back new op lower bound from 7316654177604980736 to 7316654177466765312. Current Snapshot: MvccSnapshot[applied={T|T < 7316654177604980736}]
I20260809 16:22:03.838730 7639 mvcc.cc:205] Tried to move back new op lower bound from 7316654177604980736 to 7316654177466765312. Current Snapshot: MvccSnapshot[applied={T|T < 7316654177604980736}]
I20260809 16:22:03.846511 7641 mvcc.cc:205] Tried to move back new op lower bound from 7316654177604980736 to 7316654177466765312. Current Snapshot: MvccSnapshot[applied={T|T < 7316654177604980736}]
I20260809 16:22:03.846843 6845 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45562:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 16:22:03.846943 6845 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45562:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 16:22:03.847792 6845 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b2a198ab748a408480a8aff11d6a11b4: Sending DeleteTablet for 3 replicas of tablet e7694195883841cb86e1864f532ca596
I20260809 16:22:03.848186 7480 tablet_service.cc:1514] Processing DeleteTablet for tablet e7694195883841cb86e1864f532ca596 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:22:03 UTC) from {username='kudu', principal='kudu/127.21.51.190@KRBTEST.COM'} at 127.0.0.1:37380
I20260809 16:22:03.848232 7072 tablet_service.cc:1514] Processing DeleteTablet for tablet e7694195883841cb86e1864f532ca596 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:22:03 UTC) from {username='kudu', principal='kudu/127.21.51.190@KRBTEST.COM'} at 127.0.0.1:35820
I20260809 16:22:03.848237 7283 tablet_service.cc:1514] Processing DeleteTablet for tablet e7694195883841cb86e1864f532ca596 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:22:03 UTC) from {username='kudu', principal='kudu/127.21.51.190@KRBTEST.COM'} at 127.0.0.1:43210
I20260809 16:22:03.848629 7651 tablet_replica.cc:330] T e7694195883841cb86e1864f532ca596 P cfacc3aadc964583af4135ba9b1b662a: stopping tablet replica
I20260809 16:22:03.848817 7651 raft_consensus.cc:2229] T e7694195883841cb86e1864f532ca596 P cfacc3aadc964583af4135ba9b1b662a [term 1 LEADER]: Raft consensus shutting down.
I20260809 16:22:03.848886 21710 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrNqUCn/build/release/bin/kudu with pid 6968
I20260809 16:22:03.848969 7651 raft_consensus.cc:2258] T e7694195883841cb86e1864f532ca596 P cfacc3aadc964583af4135ba9b1b662a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 16:22:03.872887 7653 tablet_replica.cc:330] T e7694195883841cb86e1864f532ca596 P 0890b7adf04c424dac299041e01ae6fe: stopping tablet replica
I20260809 16:22:03.873063 7653 raft_consensus.cc:2229] T e7694195883841cb86e1864f532ca596 P 0890b7adf04c424dac299041e01ae6fe [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 16:22:03.873202 7653 raft_consensus.cc:2258] T e7694195883841cb86e1864f532ca596 P 0890b7adf04c424dac299041e01ae6fe [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 16:22:03.873402 7654 tablet_replica.cc:330] T e7694195883841cb86e1864f532ca596 P 7f74f7c34f404fd2b8a074b249d86564: stopping tablet replica
I20260809 16:22:03.873528 7654 raft_consensus.cc:2229] T e7694195883841cb86e1864f532ca596 P 7f74f7c34f404fd2b8a074b249d86564 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 16:22:03.873632 7654 raft_consensus.cc:2258] T e7694195883841cb86e1864f532ca596 P 7f74f7c34f404fd2b8a074b249d86564 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 16:22:03.874620 7653 ts_tablet_manager.cc:1905] T e7694195883841cb86e1864f532ca596 P 0890b7adf04c424dac299041e01ae6fe: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 16:22:03.874625 7654 ts_tablet_manager.cc:1905] T e7694195883841cb86e1864f532ca596 P 7f74f7c34f404fd2b8a074b249d86564: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260809 16:22:03.875183 6814 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.21.51.129:33215 (error 108)
I20260809 16:22:03.876238 7603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:22:03.876092 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.129:33215 (local address 127.0.0.1:35836)
0809 16:22:03.876152 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0809 16:22:03.876199 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.51.129:33215: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260809 16:22:03.876353 6814 catalog_manager.cc:4584] TS cfacc3aadc964583af4135ba9b1b662a (127.21.51.129:33215): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet e7694195883841cb86e1864f532ca596: Network error: Client connection negotiation failed: client connection to 127.21.51.129:33215: connect: Connection refused (error 111)
I20260809 16:22:03.876593 21710 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrNqUCn/build/release/bin/kudu with pid 7184
I20260809 16:22:03.893564 7653 ts_tablet_manager.cc:1918] T e7694195883841cb86e1864f532ca596 P 0890b7adf04c424dac299041e01ae6fe: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 16:22:03.893694 7653 log.cc:1196] T e7694195883841cb86e1864f532ca596 P 0890b7adf04c424dac299041e01ae6fe: Deleting WAL directory at /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786292417722232-21710-0/minicluster-data/ts-2/wal/wals/e7694195883841cb86e1864f532ca596
I20260809 16:22:03.894464 7653 ts_tablet_manager.cc:1939] T e7694195883841cb86e1864f532ca596 P 0890b7adf04c424dac299041e01ae6fe: Deleting consensus metadata
I20260809 16:22:03.895080 6814 catalog_manager.cc:4841] TS 0890b7adf04c424dac299041e01ae6fe (127.21.51.131:34699): tablet e7694195883841cb86e1864f532ca596 (table test-table [id=c4ab85bfc5e048ca9aec5eb689fe041b]) successfully deleted
I20260809 16:22:03.898100 21710 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrNqUCn/build/release/bin/kudu with pid 7385
I20260809 16:22:03.898149 7603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:22:03.898017 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.130:41139 (local address 127.0.0.1:43222)
0809 16:22:03.898070 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0809 16:22:03.898108 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.51.130:41139: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
I20260809 16:22:03.908025 21710 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrNqUCn/build/release/bin/kudu with pid 6789
I20260809 16:22:03.922322 21710 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260809 16:22:04.277103 21710 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260809 16:22:04.277218 21710 mini_ranger.cc:66] Stopping Ranger...
I20260809 16:22:04.621785 21710 mini_ranger.cc:68] Stopped Ranger
2026-08-09 16:22:04.622 UTC [6160] LOG: received smart shutdown request
2026-08-09 16:22:04.625 UTC [6160] LOG: background worker "logical replication launcher" (PID 6170) exited with exit code 1
2026-08-09 16:22:04.626 UTC [6164] LOG: shutting down
2026-08-09 16:22:04.779 UTC [6160] LOG: database system is shut down
2026-08-09T16:22:04Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (47701 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292417722232-21710-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 16:22:04 dist-test-slave-s11t krb5kdc[7665](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 16:22:04 dist-test-slave-s11t krb5kdc[7665](info): set up 2 sockets
Aug 09 16:22:04 dist-test-slave-s11t krb5kdc[7665](info): commencing operation
krb5kdc: starting...
W20260809 16:22:06.134480 21710 mini_kdc.cc:121] Time spent starting KDC: real 1.217s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292417722232-21710-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292417722232-21710-0/krb5kdc/test-user.keytab.
Aug 09 16:22:06 dist-test-slave-s11t krb5kdc[7665](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292526, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T16:22:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T16:22:06Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.21.51.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.21.51.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.21.51.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292417722232-21710-0/krb5kdc/rangeradmin_127.21.51.148@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.21.51.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292417722232-21710-0/krb5kdc/rangeradmin_127.21.51.148@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.21.51.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.21.51.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.21.51.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292417722232-21710-0/krb5kdc/rangerlookup_127.21.51.148@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.21.51.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292417722232-21710-0/krb5kdc/rangerlookup_127.21.51.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.21.51.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.21.51.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.21.51.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292417722232-21710-0/krb5kdc/HTTP_127.21.51.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.21.51.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292417722232-21710-0/krb5kdc/HTTP_127.21.51.148@KRBTEST.COM.keytab.
I20260809 16:22:06.307077 21710 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292417722232-21710-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskrNqUCn/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292417722232-21710-0/minicluster-data/postgres -l logfile start
2026-08-09 16:22:08.735 UTC [7724] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-09 16:22:08.735 UTC [7724] LOG: listening on IPv4 address "127.21.51.148", port 38633
2026-08-09 16:22:08.738 UTC [7724] LOG: listening on Unix socket "/tmp/.s.PGSQL.38633"
2026-08-09 16:22:08.760 UTC [7727] LOG: database system was shut down at 2026-08-09 16:22:07 UTC
2026-08-09 16:22:08.764 UTC [7724] LOG: database system is ready to accept connections
I20260809 16:22:10.000458 21710 mini_postgres.cc:96] Postgres bound to 38633
2026-08-09 16:22:10.007 UTC [7736] FATAL: database "slave" does not exist
127.21.51.148:38633 - accepting connections
I20260809 16:22:10.008082 21710 mini_ranger.cc:161] Starting Ranger...
I20260809 16:22:10.023903 21710 mini_ranger.cc:84] Created miniranger Postgres user
I20260809 16:22:10.487648 21710 mini_ranger.cc:87] Created ranger Postgres database
I20260809 16:22:10.487775 21710 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292417722232-21710-0/minicluster-data/ranger-admin
2026-08-09 16:22:10,773 [I] DB FLAVOR :POSTGRES
2026-08-09 16:22:10,773 [I] --------- Verifying Ranger DB connection ---------
2026-08-09 16:22:10,773 [I] Checking connection..
2026-08-09 16:22:10,773 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrNqUCn/build/release/bin/postgresql.jar:/tmp/dist-test-taskrNqUCn/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.51.148:38633/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-09 16:22:11,032 [I] Checking connection passed.
2026-08-09 16:22:11,032 [I] --------- Verifying version history table ---------
2026-08-09 16:22:11,033 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrNqUCn/build/release/bin/postgresql.jar:/tmp/dist-test-taskrNqUCn/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.51.148:38633/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-09 16:22:11,267 [I] Table x_db_version_h does not exist in database ranger
2026-08-09 16:22:11,268 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrNqUCn/build/release/bin/postgresql.jar:/tmp/dist-test-taskrNqUCn/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.51.148:38633/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-09 16:22:11,531 [I] Table x_db_version_h does not exist in database ranger
2026-08-09 16:22:11,531 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-09 16:22:11,531 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrNqUCn/build/release/bin/postgresql.jar:/tmp/dist-test-taskrNqUCn/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.51.148:38633/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskrNqUCn/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-09 16:22:11.814 UTC [7835] WARNING: there is no transaction in progress
2026-08-09 16:22:11,823 [I] create_dbversion_catalog.sql file imported successfully
2026-08-09 16:22:11,823 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrNqUCn/build/release/bin/postgresql.jar:/tmp/dist-test-taskrNqUCn/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.51.148:38633/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-09 16:22:12,077 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-09 16:22:12,078 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-09 16:22:12,078 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrNqUCn/build/release/bin/postgresql.jar:/tmp/dist-test-taskrNqUCn/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.51.148:38633/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-09 16:22:12,332 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrNqUCn/build/release/bin/postgresql.jar:/tmp/dist-test-taskrNqUCn/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.51.148:38633/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-09 16:22:12,586 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrNqUCn/build/release/bin/postgresql.jar:/tmp/dist-test-taskrNqUCn/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.51.148:38633/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-s11t.c.gcp-upstream.internal','N') ;"
2026-08-09 16:22:12,843 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrNqUCn/build/release/bin/postgresql.jar:/tmp/dist-test-taskrNqUCn/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.51.148:38633/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-09 16:22:13,118 [I] Table x_portal_user does not exist in database ranger
2026-08-09 16:22:13,118 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrNqUCn/build/release/bin/postgresql.jar:/tmp/dist-test-taskrNqUCn/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.51.148:38633/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-09 16:22:13,375 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-09 16:22:13,376 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrNqUCn/build/release/bin/postgresql.jar:/tmp/dist-test-taskrNqUCn/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.51.148:38633/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-09 16:22:13,631 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrNqUCn/build/release/bin/postgresql.jar:/tmp/dist-test-taskrNqUCn/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.51.148:38633/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-09 16:22:13,874 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-09 16:22:13,874 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrNqUCn/build/release/bin/postgresql.jar:/tmp/dist-test-taskrNqUCn/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.51.148:38633/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskrNqUCn/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-09 16:22:14.809 UTC [8000] WARNING: there is no transaction in progress
2026-08-09 16:22:14.824 UTC [8000] WARNING: there is no transaction in progress
2026-08-09 16:22:14.840 UTC [8000] WARNING: there is no transaction in progress
2026-08-09 16:22:14.855 UTC [8000] WARNING: there is no transaction in progress
2026-08-09 16:22:14.870 UTC [8000] WARNING: there is no transaction in progress
2026-08-09 16:22:14.890 UTC [8000] WARNING: there is no transaction in progress
2026-08-09 16:22:14.953 UTC [8000] WARNING: there is no transaction in progress
2026-08-09 16:22:14.959 UTC [8000] WARNING: there is no transaction in progress
2026-08-09 16:22:14.973 UTC [8000] WARNING: there is no transaction in progress
2026-08-09 16:22:14.984 UTC [8000] WARNING: there is no transaction in progress
2026-08-09 16:22:14.994 UTC [8000] WARNING: there is no transaction in progress
2026-08-09 16:22:15.007 UTC [8000] WARNING: there is no transaction in progress
2026-08-09 16:22:15.016 UTC [8000] WARNING: there is no transaction in progress
2026-08-09 16:22:15.023 UTC [8000] WARNING: there is no transaction in progress
2026-08-09 16:22:15.030 UTC [8000] WARNING: there is no transaction in progress
2026-08-09 16:22:15.037 UTC [8000] WARNING: there is no transaction in progress
2026-08-09 16:22:15,658 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-09 16:22:15,658 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrNqUCn/build/release/bin/postgresql.jar:/tmp/dist-test-taskrNqUCn/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.51.148:38633/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-09 16:22:15,909 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-09 16:22:15,909 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrNqUCn/build/release/bin/postgresql.jar:/tmp/dist-test-taskrNqUCn/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.51.148:38633/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-09 16:22:16,181 [I] Table x_portal_user already exists in database 'ranger'
2026-08-09 16:22:16,182 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrNqUCn/build/release/bin/postgresql.jar:/tmp/dist-test-taskrNqUCn/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.51.148:38633/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-09 16:22:16,435 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-09 16:22:16,435 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrNqUCn/build/release/bin/postgresql.jar:/tmp/dist-test-taskrNqUCn/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.51.148:38633/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-09 16:22:16,673 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrNqUCn/build/release/bin/postgresql.jar:/tmp/dist-test-taskrNqUCn/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.51.148:38633/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-09 16:22:16,909 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrNqUCn/build/release/bin/postgresql.jar:/tmp/dist-test-taskrNqUCn/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.51.148:38633/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-s11t.c.gcp-upstream.internal';"
2026-08-09 16:22:17,165 [I] CORE_DB_SCHEMA import status has been updated
2026-08-09 16:22:17,166 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrNqUCn/build/release/bin/postgresql.jar:/tmp/dist-test-taskrNqUCn/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.51.148:38633/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-09 16:22:17,415 [I] DB_PATCHES have already been applied
I20260809 16:22:17.420540 21710 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292417722232-21710-0/minicluster-data/ranger-admin:/tmp/dist-test-taskrNqUCn/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskrNqUCn/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskrNqUCn/build/release/bin/postgresql.jar:/tmp/dist-test-taskrNqUCn/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260809 16:22:17.420662 21710 mini_ranger.cc:193] Using host: 127.21.51.148
I20260809 16:22:17.424945 21710 mini_ranger.cc:227] Ranger admin URL: http://127.21.51.148:60327
Aug 09, 2026 4:22:17 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 09, 2026 4:22:17 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 09, 2026 4:22:17 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292417722232-21710-0/minicluster-data/ranger-admin/ews/webapp
Aug 09, 2026 4:22:17 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292417722232-21710-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 09, 2026 4:22:17 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292417722232-21710-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 09, 2026 4:22:18 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292417722232-21710-0/minicluster-data/ranger-admin/ews/webapp].
Aug 09, 2026 4:22:18 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260809 16:22:18.769421 21710 mini_ranger.cc:160] Time spent starting Ranger: real 8.761s user 0.001s sys 0.009s
Aug 09, 2026 4:22:38 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 09, 2026 4:22:38 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 09, 2026 4:22:38 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 09, 2026 4:22:38 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 09, 2026 4:22:39 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 09, 2026 4:22:39 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 09, 2026 4:22:39 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 09, 2026 4:22:39 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 09, 2026 4:22:39 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 09, 2026 4:22:39 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 09, 2026 4:22:39 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 09, 2026 4:22:39 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 09, 2026 4:22:39 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 09, 2026 4:22:39 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 09, 2026 4:22:39 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 09, 2026 4:22:39 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 09, 2026 4:22:39 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 09, 2026 4:22:39 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260809 16:22:40.546446 21710 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.21.51.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.21.51.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.21.51.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292417722232-21710-0/krb5kdc/rangerkms_127.21.51.148@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.21.51.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292417722232-21710-0/krb5kdc/rangerkms_127.21.51.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.21.51.148@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.21.51.148@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.21.51.148@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292417722232-21710-0/krb5kdc/HTTP_127.21.51.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.21.51.148@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292417722232-21710-0/krb5kdc/HTTP_127.21.51.148@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 09 16:22:40 dist-test-slave-s11t krb5kdc[7665](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292560, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260809 16:22:40.630554 21710 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260809 16:22:40.641170 21710 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260809 16:22:41.391762 21710 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260809 16:22:41.391852 21710 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292417722232-21710-0/minicluster-data/ranger-kms
I20260809 16:22:41.391872 21710 mini_ranger_kms.cc:227] Using postgres at 127.21.51.148:38633
2026-08-09 16:22:41,567 [I] DB FLAVOR :POSTGRES
2026-08-09 16:22:41,567 [I] --------- Verifying Ranger DB connection ---------
2026-08-09 16:22:41,567 [I] Checking connection
2026-08-09 16:22:41,567 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrNqUCn/build/release/bin/postgresql.jar:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292417722232-21710-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.51.148:38633/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-09 16:22:41,830 [I] connection success
2026-08-09 16:22:41,830 [I] --------- Verifying Ranger DB tables ---------
2026-08-09 16:22:41,831 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-09 16:22:41,831 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrNqUCn/build/release/bin/postgresql.jar:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292417722232-21710-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.51.148:38633/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-09 16:22:42,074 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-09 16:22:42,074 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-09 16:22:42,074 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-09 16:22:42,075 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrNqUCn/build/release/bin/postgresql.jar:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292417722232-21710-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.51.148:38633/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292417722232-21710-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-09 16:22:42,342 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260809 16:22:42.830928 21710 mini_ranger_kms.cc:310] Created kms service
I20260809 16:22:43.020366 21710 mini_ranger_kms.cc:326] Created kudu user
I20260809 16:22:43.068897 21710 mini_ranger_kms.cc:343] Created rangerkms user
Aug 09 16:22:43 dist-test-slave-s11t krb5kdc[7665](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292560, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.21.51.148@KRBTEST.COM
I20260809 16:22:43.484542 21710 mini_ranger_kms.cc:384] Added ranger policy
I20260809 16:22:43.484790 21710 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292417722232-21710-0/minicluster-data/ranger-kms:/tmp/dist-test-taskrNqUCn/build/release/bin/postgresql.jar:/tmp/dist-test-taskrNqUCn/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskrNqUCn/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260809 16:22:43.484829 21710 mini_ranger_kms.cc:240] Using host: 127.21.51.148
I20260809 16:22:43.489122 21710 mini_ranger_kms.cc:276] Ranger KMS PID: 8287
I20260809 16:22:43.489199 21710 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.21.51.148:39707
Aug 09, 2026 4:22:44 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 09, 2026 4:22:44 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 09, 2026 4:22:44 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292417722232-21710-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 09, 2026 4:22:44 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292417722232-21710-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 09, 2026 4:22:45 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292417722232-21710-0/minicluster-data/ranger-kms/ews/webapp].
Aug 09, 2026 4:22:45 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 09 16:22:45 dist-test-slave-s11t krb5kdc[7665](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786292565, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.21.51.148@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:22:46.200165 21710 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.570s user 0.011s sys 0.149s
I20260809 16:22:46.200264 21710 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260809 16:22:46.200312 21710 mini_ranger_kms.cc:401] 127.21.51.148:39707/kms/v1/keys
Aug 09 16:22:46 dist-test-slave-s11t krb5kdc[7665](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292565, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.21.51.148@KRBTEST.COM for HTTP/127.21.51.148@KRBTEST.COM
Aug 09 16:22:46 dist-test-slave-s11t krb5kdc[7665](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292565, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.21.51.148@KRBTEST.COM for HTTP/127.21.51.148@KRBTEST.COM
Aug 09 16:22:49 dist-test-slave-s11t krb5kdc[7665](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292569, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.21.51.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.51.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.51.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.190.keytab.
Entry for principal kudu/127.21.51.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.190.keytab.
I20260809 16:22:49.143442 21710 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrNqUCn/build/release/bin/kudu
/tmp/dist-test-taskrNqUCn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292417722232-21710-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292417722232-21710-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292417722232-21710-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292417722232-21710-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.51.190:38333
--webserver_interface=127.21.51.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.190.keytab
--principal=kudu/127.21.51.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.51.148:39109
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.51.190:38333
--ranger_config_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292417722232-21710-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.21.51.148:39707/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292417722232-21710-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292417722232-21710-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292417722232-21710-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:22:49.229339 8367 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:22:49.229565 8367 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:22:49.229601 8367 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:22:49.230818 8367 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 16:22:49.230881 8367 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:22:49.230907 8367 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:22:49.230942 8367 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 16:22:49.230971 8367 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 16:22:49.232537 8367 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.51.148:39109
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292417722232-21710-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292417722232-21710-0/minicluster-data/master-0/wal
--ranger_kms_url=127.21.51.148:39707/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.21.51.190:38333
--ranger_config_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292417722232-21710-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.190.keytab
--principal=kudu/127.21.51.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.51.190:38333
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292417722232-21710-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.51.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8367
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292417722232-21710-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:22:49.232837 8367 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:22:49.233078 8367 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:22:49.238111 8374 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:22:49.238118 8376 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:22:49.238118 8373 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:22:49.238477 8367 server_base.cc:1034] running on GCE node
I20260809 16:22:49.238740 8367 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:22:49.239100 8367 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:22:49.240257 8367 hybrid_clock.cc:648] HybridClock initialized: now 1786292569240231 us; error 42 us; skew 500 ppm
Aug 09 16:22:49 dist-test-slave-s11t krb5kdc[7665](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292569, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.51.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:22:49.246873 8367 init.cc:377] Logged in from keytab as kudu/127.21.51.190@KRBTEST.COM (short username kudu)
I20260809 16:22:49.247413 8367 webserver.cc:460] Webserver started at http://127.21.51.190:40711/ using document root <none> and password file <none>
I20260809 16:22:49.247682 8367 fs_manager.cc:359] Metadata directory not provided
I20260809 16:22:49.247740 8367 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:22:49.247866 8367 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 16:22:49 dist-test-slave-s11t krb5kdc[7665](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292569, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.51.190@KRBTEST.COM for HTTP/127.21.51.148@KRBTEST.COM
I20260809 16:22:49.294198 8367 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292417722232-21710-0/minicluster-data/master-0/data/instance:
uuid: "860b05f8620c4cc2a5be9cec7d0e0fcb"
format_stamp: "Formatted at 2026-08-09 16:22:49 on dist-test-slave-s11t"
server_key: "3d1ed0c397b4eb09102098e541307065"
server_key_iv: "873f5335787222f984b3709845e19c08"
server_key_version: "kuduclusterkey@0"
I20260809 16:22:49.294625 8367 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292417722232-21710-0/minicluster-data/master-0/wal/instance:
uuid: "860b05f8620c4cc2a5be9cec7d0e0fcb"
format_stamp: "Formatted at 2026-08-09 16:22:49 on dist-test-slave-s11t"
server_key: "3d1ed0c397b4eb09102098e541307065"
server_key_iv: "873f5335787222f984b3709845e19c08"
server_key_version: "kuduclusterkey@0"
I20260809 16:22:49.296114 8367 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260809 16:22:49.311240 8385 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:22:49.311506 8367 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:22:49.311606 8367 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292417722232-21710-0/minicluster-data/master-0/data,/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292417722232-21710-0/minicluster-data/master-0/wal
uuid: "860b05f8620c4cc2a5be9cec7d0e0fcb"
format_stamp: "Formatted at 2026-08-09 16:22:49 on dist-test-slave-s11t"
server_key: "3d1ed0c397b4eb09102098e541307065"
server_key_iv: "873f5335787222f984b3709845e19c08"
server_key_version: "kuduclusterkey@0"
I20260809 16:22:49.311695 8367 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292417722232-21710-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292417722232-21710-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292417722232-21710-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:22:49.328096 8367 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:22:49.330137 8367 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:22:49.330303 8367 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:22:49.335783 8367 rpc_server.cc:307] RPC server started. Bound to: 127.21.51.190:38333
I20260809 16:22:49.335856 8469 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.51.190:38333 every 8 connection(s)
I20260809 16:22:49.336225 8367 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292417722232-21710-0/minicluster-data/master-0/data/info.pb
I20260809 16:22:49.337024 8470 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:22:49.339404 8470 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 860b05f8620c4cc2a5be9cec7d0e0fcb: Bootstrap starting.
I20260809 16:22:49.339854 21710 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrNqUCn/build/release/bin/kudu as pid 8367
I20260809 16:22:49.339965 21710 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292417722232-21710-0/minicluster-data/master-0/wal/instance
I20260809 16:22:49.340153 21710 external_mini_cluster.cc:1428] Setting key 1734fae9bd9ec1233a0ab2cf6b1a5a4f
I20260809 16:22:49.340236 8470 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 860b05f8620c4cc2a5be9cec7d0e0fcb: Neither blocks nor log segments found. Creating new log.
I20260809 16:22:49.340626 8470 log.cc:824] T 00000000000000000000000000000000 P 860b05f8620c4cc2a5be9cec7d0e0fcb: Log is configured to *not* fsync() on all Append() calls
I20260809 16:22:49.341471 8470 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 860b05f8620c4cc2a5be9cec7d0e0fcb: No bootstrap required, opened a new log
I20260809 16:22:49.343441 8470 raft_consensus.cc:348] T 00000000000000000000000000000000 P 860b05f8620c4cc2a5be9cec7d0e0fcb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "860b05f8620c4cc2a5be9cec7d0e0fcb" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 38333 } }
I20260809 16:22:49.343606 8470 raft_consensus.cc:374] T 00000000000000000000000000000000 P 860b05f8620c4cc2a5be9cec7d0e0fcb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:22:49.343653 8470 raft_consensus.cc:729] T 00000000000000000000000000000000 P 860b05f8620c4cc2a5be9cec7d0e0fcb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 860b05f8620c4cc2a5be9cec7d0e0fcb, State: Initialized, Role: FOLLOWER
I20260809 16:22:49.343786 8470 consensus_queue.cc:260] T 00000000000000000000000000000000 P 860b05f8620c4cc2a5be9cec7d0e0fcb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "860b05f8620c4cc2a5be9cec7d0e0fcb" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 38333 } }
I20260809 16:22:49.343870 8470 raft_consensus.cc:388] T 00000000000000000000000000000000 P 860b05f8620c4cc2a5be9cec7d0e0fcb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 16:22:49.343911 8470 raft_consensus.cc:482] T 00000000000000000000000000000000 P 860b05f8620c4cc2a5be9cec7d0e0fcb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 16:22:49.343951 8470 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 860b05f8620c4cc2a5be9cec7d0e0fcb [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:22:49.344580 8470 raft_consensus.cc:504] T 00000000000000000000000000000000 P 860b05f8620c4cc2a5be9cec7d0e0fcb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "860b05f8620c4cc2a5be9cec7d0e0fcb" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 38333 } }
I20260809 16:22:49.344779 8470 leader_election.cc:302] T 00000000000000000000000000000000 P 860b05f8620c4cc2a5be9cec7d0e0fcb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 860b05f8620c4cc2a5be9cec7d0e0fcb; no voters:
I20260809 16:22:49.344988 8470 leader_election.cc:288] T 00000000000000000000000000000000 P 860b05f8620c4cc2a5be9cec7d0e0fcb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 16:22:49.345299 8470 sys_catalog.cc:564] T 00000000000000000000000000000000 P 860b05f8620c4cc2a5be9cec7d0e0fcb [sys.catalog]: configured and running, proceeding with master startup.
I20260809 16:22:49.345731 8480 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 860b05f8620c4cc2a5be9cec7d0e0fcb [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:22:49.345892 8480 raft_consensus.cc:686] T 00000000000000000000000000000000 P 860b05f8620c4cc2a5be9cec7d0e0fcb [term 1 LEADER]: Becoming Leader. State: Replica: 860b05f8620c4cc2a5be9cec7d0e0fcb, State: Running, Role: LEADER
I20260809 16:22:49.345995 8480 consensus_queue.cc:237] T 00000000000000000000000000000000 P 860b05f8620c4cc2a5be9cec7d0e0fcb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "860b05f8620c4cc2a5be9cec7d0e0fcb" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 38333 } }
I20260809 16:22:49.346735 8480 sys_catalog.cc:455] T 00000000000000000000000000000000 P 860b05f8620c4cc2a5be9cec7d0e0fcb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 860b05f8620c4cc2a5be9cec7d0e0fcb. Latest consensus state: current_term: 1 leader_uuid: "860b05f8620c4cc2a5be9cec7d0e0fcb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "860b05f8620c4cc2a5be9cec7d0e0fcb" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 38333 } } }
I20260809 16:22:49.346923 8480 sys_catalog.cc:458] T 00000000000000000000000000000000 P 860b05f8620c4cc2a5be9cec7d0e0fcb [sys.catalog]: This master's current role is: LEADER
I20260809 16:22:49.346743 8484 sys_catalog.cc:455] T 00000000000000000000000000000000 P 860b05f8620c4cc2a5be9cec7d0e0fcb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "860b05f8620c4cc2a5be9cec7d0e0fcb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "860b05f8620c4cc2a5be9cec7d0e0fcb" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 38333 } } }
I20260809 16:22:49.347044 8484 sys_catalog.cc:458] T 00000000000000000000000000000000 P 860b05f8620c4cc2a5be9cec7d0e0fcb [sys.catalog]: This master's current role is: LEADER
I20260809 16:22:49.347098 8470 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292417722232-21710-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260809 16:22:49.347368 8498 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
Aug 09 16:22:49 dist-test-slave-s11t krb5kdc[7665](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292569, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.51.190@KRBTEST.COM
I20260809 16:22:49.351516 8498 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 16:22:49.355008 8473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:22:49.340755 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39740 (local address 127.21.51.190:38333)
0809 16:22:49.340976 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0809 16:22:49.340986 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 16:22:49.341019 (+ 33us) server_negotiation.cc:407] Connection header received
0809 16:22:49.341584 (+ 565us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:22:49.341595 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:22:49.341660 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:22:49.341817 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:22:49.342665 (+ 848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:22:49.344558 (+ 1893us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:22:49.345745 (+ 1187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:22:49.346728 (+ 983us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:22:49.346837 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:22:49.350409 (+ 3572us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:22:49.350479 (+ 70us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:22:49.350480 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:22:49.350612 (+ 132us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:22:49.353507 (+ 2895us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:22:49.353857 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:22:49.353860 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:22:49.353861 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:22:49.353950 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:22:49.354157 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:22:49.354158 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:22:49.354158 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:22:49.354411 (+ 253us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:22:49.354593 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0809 16:22:49.354684 (+ 91us) server_negotiation.cc:299] Negotiation successful
0809 16:22:49.354850 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":172,"thread_start_us":97,"threads_started":1}
I20260809 16:22:49.357211 8498 catalog_manager.cc:1347] Generated new cluster ID: 940f46a6dbb04cc9b0eaed9b671e5af3
I20260809 16:22:49.357260 8498 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 16:22:49.371640 8498 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 16:22:49.372388 8498 catalog_manager.cc:1504] Loading token signing keys...
I20260809 16:22:49.374840 8498 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 860b05f8620c4cc2a5be9cec7d0e0fcb: Generated new TSK 0
I20260809 16:22:49.375017 8498 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.21.51.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.51.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.51.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.129.keytab.
Entry for principal kudu/127.21.51.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.129.keytab.
I20260809 16:22:50.091985 21710 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrNqUCn/build/release/bin/kudu
/tmp/dist-test-taskrNqUCn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292417722232-21710-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292417722232-21710-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292417722232-21710-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292417722232-21710-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.21.51.148:39707/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.51.129:0
--local_ip_for_outbound_sockets=127.21.51.129
--webserver_interface=127.21.51.129
--webserver_port=0
--tserver_master_addrs=127.21.51.190:38333
--keytab_file=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.129.keytab
--principal=kudu/127.21.51.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.51.148:39109
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292417722232-21710-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292417722232-21710-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292417722232-21710-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:22:50.218674 8549 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:22:50.218968 8549 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:22:50.219002 8549 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:22:50.220670 8549 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:22:50.220750 8549 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:22:50.220788 8549 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.51.129
I20260809 16:22:50.223100 8549 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.51.148:39109
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292417722232-21710-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292417722232-21710-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.21.51.148:39707/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.129.keytab
--principal=kudu/127.21.51.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.51.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292417722232-21710-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.51.129
--webserver_port=0
--tserver_master_addrs=127.21.51.190:38333
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8549
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.51.129
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292417722232-21710-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:22:50.223352 8549 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:22:50.223621 8549 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:22:50.230641 8565 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:22:50.230654 8562 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:22:50.230928 8560 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:22:50.231688 8549 server_base.cc:1034] running on GCE node
I20260809 16:22:50.231899 8549 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:22:50.232190 8549 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:22:50.233392 8549 hybrid_clock.cc:648] HybridClock initialized: now 1786292570233387 us; error 44 us; skew 500 ppm
Aug 09 16:22:50 dist-test-slave-s11t krb5kdc[7665](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292570, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.51.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:22:50.236529 8549 init.cc:377] Logged in from keytab as kudu/127.21.51.129@KRBTEST.COM (short username kudu)
I20260809 16:22:50.237073 8549 webserver.cc:460] Webserver started at http://127.21.51.129:37731/ using document root <none> and password file <none>
I20260809 16:22:50.237334 8549 fs_manager.cc:359] Metadata directory not provided
I20260809 16:22:50.237419 8549 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:22:50.237566 8549 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 16:22:50 dist-test-slave-s11t krb5kdc[7665](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292570, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.51.129@KRBTEST.COM for HTTP/127.21.51.148@KRBTEST.COM
I20260809 16:22:50.253785 8549 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292417722232-21710-0/minicluster-data/ts-0/data/instance:
uuid: "22ecd31dbeeb44ad8fb4a150f0af7665"
format_stamp: "Formatted at 2026-08-09 16:22:50 on dist-test-slave-s11t"
server_key: "b3fb8b09c89cd591c01e85e1b18893f4"
server_key_iv: "aa1f3d19a8639f6daea85edfb0d3b58e"
server_key_version: "kuduclusterkey@0"
I20260809 16:22:50.254244 8549 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292417722232-21710-0/minicluster-data/ts-0/wal/instance:
uuid: "22ecd31dbeeb44ad8fb4a150f0af7665"
format_stamp: "Formatted at 2026-08-09 16:22:50 on dist-test-slave-s11t"
server_key: "b3fb8b09c89cd591c01e85e1b18893f4"
server_key_iv: "aa1f3d19a8639f6daea85edfb0d3b58e"
server_key_version: "kuduclusterkey@0"
I20260809 16:22:50.259172 8549 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.001s
I20260809 16:22:50.275915 8576 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:22:50.276211 8549 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.000s
I20260809 16:22:50.276340 8549 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292417722232-21710-0/minicluster-data/ts-0/data,/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292417722232-21710-0/minicluster-data/ts-0/wal
uuid: "22ecd31dbeeb44ad8fb4a150f0af7665"
format_stamp: "Formatted at 2026-08-09 16:22:50 on dist-test-slave-s11t"
server_key: "b3fb8b09c89cd591c01e85e1b18893f4"
server_key_iv: "aa1f3d19a8639f6daea85edfb0d3b58e"
server_key_version: "kuduclusterkey@0"
I20260809 16:22:50.276448 8549 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292417722232-21710-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292417722232-21710-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292417722232-21710-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:22:50.288892 8549 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:22:50.291231 8549 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:22:50.291374 8549 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:22:50.291620 8549 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:22:50.291998 8549 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:22:50.292057 8549 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:22:50.292088 8549 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:22:50.292157 8549 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:22:50.299969 8549 rpc_server.cc:307] RPC server started. Bound to: 127.21.51.129:44557
I20260809 16:22:50.300038 8763 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.51.129:44557 every 8 connection(s)
I20260809 16:22:50.300484 8549 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292417722232-21710-0/minicluster-data/ts-0/data/info.pb
I20260809 16:22:50.307505 21710 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrNqUCn/build/release/bin/kudu as pid 8549
I20260809 16:22:50.307610 21710 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292417722232-21710-0/minicluster-data/ts-0/wal/instance
I20260809 16:22:50.307807 21710 external_mini_cluster.cc:1428] Setting key 99d1a123e2b6ffbbea34afcb9ba2b9de
Aug 09 16:22:50 dist-test-slave-s11t krb5kdc[7665](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292570, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.51.129@KRBTEST.COM for kudu/127.21.51.190@KRBTEST.COM
I20260809 16:22:50.314075 8774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:22:50.301420 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.51.129:36687 (local address 127.21.51.190:38333)
0809 16:22:50.304853 (+ 3433us) server_negotiation.cc:206] Beginning negotiation
0809 16:22:50.304856 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 16:22:50.304866 (+ 10us) server_negotiation.cc:407] Connection header received
0809 16:22:50.304896 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:22:50.304897 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:22:50.304911 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:22:50.304995 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:22:50.305755 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:22:50.306550 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:22:50.308097 (+ 1547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:22:50.308395 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:22:50.308467 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:22:50.310900 (+ 2433us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:22:50.310901 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:22:50.310902 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:22:50.310909 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:22:50.312407 (+ 1498us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:22:50.312930 (+ 523us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:22:50.312932 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:22:50.312932 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:22:50.312992 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:22:50.313282 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:22:50.313284 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:22:50.313284 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:22:50.313439 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:22:50.313520 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0809 16:22:50.313885 (+ 365us) server_negotiation.cc:299] Negotiation successful
0809 16:22:50.313979 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3397,"thread_start_us":86,"threads_started":1}
I20260809 16:22:50.314261 8773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:22:50.301389 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:38333 (local address 127.21.51.129:36687)
0809 16:22:50.301593 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0809 16:22:50.301640 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0809 16:22:50.302276 (+ 636us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:22:50.305098 (+ 2822us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:22:50.305102 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:22:50.305347 (+ 245us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:22:50.305630 (+ 283us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:22:50.305636 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:22:50.306665 (+ 1029us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:22:50.306666 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:22:50.307989 (+ 1323us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:22:50.307992 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:22:50.308554 (+ 562us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:22:50.308555 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:22:50.308734 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:22:50.309344 (+ 610us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:22:50.309354 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:22:50.310801 (+ 1447us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:22:50.312509 (+ 1708us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:22:50.312516 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:22:50.312518 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:22:50.312835 (+ 317us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:22:50.313054 (+ 219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:22:50.313054 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:22:50.313055 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:22:50.313194 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:22:50.313526 (+ 332us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:22:50.313534 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:22:50.313705 (+ 171us) client_negotiation.cc:769] Sending connection context
0809 16:22:50.313839 (+ 134us) client_negotiation.cc:240] Negotiation successful
0809 16:22:50.314009 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":95,"threads_started":1}
I20260809 16:22:50.316174 8770 heartbeater.cc:347] Connected to a master server at 127.21.51.190:38333
I20260809 16:22:50.316303 8770 heartbeater.cc:464] Registering TS with master...
I20260809 16:22:50.316511 8770 heartbeater.cc:511] Master 127.21.51.190:38333 requested a full tablet report, sending...
I20260809 16:22:50.322890 8417 ts_manager.cc:194] Registered new tserver with Master: 22ecd31dbeeb44ad8fb4a150f0af7665 (127.21.51.129:44557)
I20260809 16:22:50.323825 8417 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.51.129@KRBTEST.COM'} at 127.21.51.129:36687
WARNING: no policy specified for kudu/127.21.51.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.51.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.51.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.130.keytab.
Entry for principal kudu/127.21.51.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.130.keytab.
I20260809 16:22:50.351931 21710 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrNqUCn/build/release/bin/kudu
/tmp/dist-test-taskrNqUCn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292417722232-21710-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292417722232-21710-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292417722232-21710-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292417722232-21710-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.21.51.148:39707/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.51.130:0
--local_ip_for_outbound_sockets=127.21.51.130
--webserver_interface=127.21.51.130
--webserver_port=0
--tserver_master_addrs=127.21.51.190:38333
--keytab_file=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.130.keytab
--principal=kudu/127.21.51.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.51.148:39109
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292417722232-21710-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292417722232-21710-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292417722232-21710-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:22:50.470000 8784 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:22:50.470291 8784 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:22:50.470337 8784 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:22:50.472072 8784 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:22:50.472139 8784 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:22:50.472182 8784 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.51.130
I20260809 16:22:50.474412 8784 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.51.148:39109
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292417722232-21710-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292417722232-21710-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.21.51.148:39707/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.130.keytab
--principal=kudu/127.21.51.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.51.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292417722232-21710-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.51.130
--webserver_port=0
--tserver_master_addrs=127.21.51.190:38333
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8784
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.51.130
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292417722232-21710-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:22:50.474680 8784 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:22:50.475044 8784 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:22:50.481820 8800 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:22:50.481848 8797 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:22:50.481866 8793 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:22:50.483121 8784 server_base.cc:1034] running on GCE node
I20260809 16:22:50.483361 8784 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:22:50.483645 8784 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:22:50.484789 8784 hybrid_clock.cc:648] HybridClock initialized: now 1786292570484763 us; error 39 us; skew 500 ppm
Aug 09 16:22:50 dist-test-slave-s11t krb5kdc[7665](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292570, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.51.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:22:50.488046 8784 init.cc:377] Logged in from keytab as kudu/127.21.51.130@KRBTEST.COM (short username kudu)
I20260809 16:22:50.488600 8784 webserver.cc:460] Webserver started at http://127.21.51.130:43149/ using document root <none> and password file <none>
I20260809 16:22:50.488931 8784 fs_manager.cc:359] Metadata directory not provided
I20260809 16:22:50.489003 8784 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:22:50.489151 8784 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 16:22:50 dist-test-slave-s11t krb5kdc[7665](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292570, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.51.130@KRBTEST.COM for HTTP/127.21.51.148@KRBTEST.COM
Aug 09 16:22:50 dist-test-slave-s11t krb5kdc[7665](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786292570, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.51.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:22:50.508369 8784 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292417722232-21710-0/minicluster-data/ts-1/data/instance:
uuid: "06717649045b4dc685420de2d8faec74"
format_stamp: "Formatted at 2026-08-09 16:22:50 on dist-test-slave-s11t"
server_key: "dfa6a5c5fb641b9aab8570dfbfd24422"
server_key_iv: "ad7492344a479b25a12bba94c1b5a492"
server_key_version: "kuduclusterkey@0"
I20260809 16:22:50.509089 8784 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292417722232-21710-0/minicluster-data/ts-1/wal/instance:
uuid: "06717649045b4dc685420de2d8faec74"
format_stamp: "Formatted at 2026-08-09 16:22:50 on dist-test-slave-s11t"
server_key: "dfa6a5c5fb641b9aab8570dfbfd24422"
server_key_iv: "ad7492344a479b25a12bba94c1b5a492"
server_key_version: "kuduclusterkey@0"
I20260809 16:22:50.511675 8784 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.001s sys 0.000s
I20260809 16:22:50.524195 8812 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:22:50.524443 8784 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 16:22:50.524541 8784 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292417722232-21710-0/minicluster-data/ts-1/data,/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292417722232-21710-0/minicluster-data/ts-1/wal
uuid: "06717649045b4dc685420de2d8faec74"
format_stamp: "Formatted at 2026-08-09 16:22:50 on dist-test-slave-s11t"
server_key: "dfa6a5c5fb641b9aab8570dfbfd24422"
server_key_iv: "ad7492344a479b25a12bba94c1b5a492"
server_key_version: "kuduclusterkey@0"
I20260809 16:22:50.524624 8784 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292417722232-21710-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292417722232-21710-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292417722232-21710-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:22:50.538501 8784 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:22:50.541307 8784 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:22:50.541512 8784 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:22:50.541764 8784 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:22:50.542098 8784 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:22:50.542147 8784 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:22:50.542178 8784 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:22:50.542202 8784 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:22:50.549810 8784 rpc_server.cc:307] RPC server started. Bound to: 127.21.51.130:43121
I20260809 16:22:50.550299 8784 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292417722232-21710-0/minicluster-data/ts-1/data/info.pb
I20260809 16:22:50.551029 8975 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.51.130:43121 every 8 connection(s)
Aug 09 16:22:50 dist-test-slave-s11t krb5kdc[7665](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292570, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.51.130@KRBTEST.COM for kudu/127.21.51.190@KRBTEST.COM
I20260809 16:22:50.559541 21710 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrNqUCn/build/release/bin/kudu as pid 8784
I20260809 16:22:50.559661 21710 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292417722232-21710-0/minicluster-data/ts-1/wal/instance
I20260809 16:22:50.559875 21710 external_mini_cluster.cc:1428] Setting key f58c8fefd14e31b081af5af595f86e08
I20260809 16:22:50.560106 8774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:22:50.551473 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.51.130:38689 (local address 127.21.51.190:38333)
0809 16:22:50.551557 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0809 16:22:50.551560 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 16:22:50.551825 (+ 265us) server_negotiation.cc:407] Connection header received
0809 16:22:50.552345 (+ 520us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:22:50.552346 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:22:50.552359 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:22:50.552425 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:22:50.553063 (+ 638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:22:50.553573 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:22:50.554577 (+ 1004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:22:50.554789 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:22:50.554878 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:22:50.557220 (+ 2342us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:22:50.557221 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:22:50.557222 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:22:50.557228 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:22:50.558624 (+ 1396us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:22:50.559048 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:22:50.559050 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:22:50.559050 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:22:50.559109 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:22:50.559371 (+ 262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:22:50.559372 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:22:50.559372 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:22:50.559487 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:22:50.559564 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0809 16:22:50.559915 (+ 351us) server_negotiation.cc:299] Negotiation successful
0809 16:22:50.560009 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260809 16:22:50.560369 8978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:22:50.551449 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:38333 (local address 127.21.51.130:38689)
0809 16:22:50.551701 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0809 16:22:50.551737 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0809 16:22:50.552239 (+ 502us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:22:50.552498 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:22:50.552501 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:22:50.552739 (+ 238us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:22:50.552971 (+ 232us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:22:50.552976 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:22:50.553675 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:22:50.553676 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:22:50.554481 (+ 805us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:22:50.554483 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:22:50.554902 (+ 419us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:22:50.554903 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:22:50.555020 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:22:50.555570 (+ 550us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:22:50.555578 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:22:50.557109 (+ 1531us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:22:50.558734 (+ 1625us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:22:50.558741 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:22:50.558743 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:22:50.558969 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:22:50.559180 (+ 211us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:22:50.559180 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:22:50.559181 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:22:50.559295 (+ 114us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:22:50.559555 (+ 260us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:22:50.559562 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:22:50.559756 (+ 194us) client_negotiation.cc:769] Sending connection context
0809 16:22:50.559881 (+ 125us) client_negotiation.cc:240] Negotiation successful
0809 16:22:50.560026 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":218,"thread_start_us":108,"threads_started":1}
I20260809 16:22:50.569047 8976 heartbeater.cc:347] Connected to a master server at 127.21.51.190:38333
I20260809 16:22:50.569175 8976 heartbeater.cc:464] Registering TS with master...
I20260809 16:22:50.569406 8976 heartbeater.cc:511] Master 127.21.51.190:38333 requested a full tablet report, sending...
I20260809 16:22:50.569872 8417 ts_manager.cc:194] Registered new tserver with Master: 06717649045b4dc685420de2d8faec74 (127.21.51.130:43121)
I20260809 16:22:50.570405 8417 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.51.130@KRBTEST.COM'} at 127.21.51.130:38689
WARNING: no policy specified for kudu/127.21.51.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.51.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.51.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.131.keytab.
Entry for principal kudu/127.21.51.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.131.keytab.
I20260809 16:22:50.592027 21710 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrNqUCn/build/release/bin/kudu
/tmp/dist-test-taskrNqUCn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292417722232-21710-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292417722232-21710-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292417722232-21710-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292417722232-21710-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.21.51.148:39707/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.51.131:0
--local_ip_for_outbound_sockets=127.21.51.131
--webserver_interface=127.21.51.131
--webserver_port=0
--tserver_master_addrs=127.21.51.190:38333
--keytab_file=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.131.keytab
--principal=kudu/127.21.51.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.51.148:39109
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292417722232-21710-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292417722232-21710-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292417722232-21710-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:22:50.716632 8982 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:22:50.716955 8982 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:22:50.716992 8982 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:22:50.718704 8982 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:22:50.718771 8982 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:22:50.718812 8982 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.51.131
I20260809 16:22:50.721074 8982 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.51.148:39109
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292417722232-21710-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292417722232-21710-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.21.51.148:39707/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.131.keytab
--principal=kudu/127.21.51.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.51.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292417722232-21710-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.51.131
--webserver_port=0
--tserver_master_addrs=127.21.51.190:38333
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8982
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.51.131
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292417722232-21710-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:22:50.721372 8982 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:22:50.721688 8982 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:22:50.728132 8994 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:22:50.728338 8993 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:22:50.728148 8996 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:22:50.729411 8982 server_base.cc:1034] running on GCE node
I20260809 16:22:50.729662 8982 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:22:50.729964 8982 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:22:50.731153 8982 hybrid_clock.cc:648] HybridClock initialized: now 1786292570731123 us; error 47 us; skew 500 ppm
Aug 09 16:22:50 dist-test-slave-s11t krb5kdc[7665](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292570, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.51.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:22:50.734511 8982 init.cc:377] Logged in from keytab as kudu/127.21.51.131@KRBTEST.COM (short username kudu)
I20260809 16:22:50.735051 8982 webserver.cc:460] Webserver started at http://127.21.51.131:34663/ using document root <none> and password file <none>
I20260809 16:22:50.735329 8982 fs_manager.cc:359] Metadata directory not provided
I20260809 16:22:50.735385 8982 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:22:50.735512 8982 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 16:22:50 dist-test-slave-s11t krb5kdc[7665](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292570, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.51.131@KRBTEST.COM for HTTP/127.21.51.148@KRBTEST.COM
I20260809 16:22:50.751642 8982 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292417722232-21710-0/minicluster-data/ts-2/data/instance:
uuid: "ef84aefd53974155b0d5ffa6d691b55f"
format_stamp: "Formatted at 2026-08-09 16:22:50 on dist-test-slave-s11t"
server_key: "03d46e5a03302c0a4e0834040ce4ca94"
server_key_iv: "9aa4f98d336e25d62e8c4e28112e6ba9"
server_key_version: "kuduclusterkey@0"
I20260809 16:22:50.752128 8982 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292417722232-21710-0/minicluster-data/ts-2/wal/instance:
uuid: "ef84aefd53974155b0d5ffa6d691b55f"
format_stamp: "Formatted at 2026-08-09 16:22:50 on dist-test-slave-s11t"
server_key: "03d46e5a03302c0a4e0834040ce4ca94"
server_key_iv: "9aa4f98d336e25d62e8c4e28112e6ba9"
server_key_version: "kuduclusterkey@0"
I20260809 16:22:50.755118 8982 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.003s
I20260809 16:22:50.773281 9008 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:22:50.775808 8982 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.000s
I20260809 16:22:50.775941 8982 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292417722232-21710-0/minicluster-data/ts-2/data,/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292417722232-21710-0/minicluster-data/ts-2/wal
uuid: "ef84aefd53974155b0d5ffa6d691b55f"
format_stamp: "Formatted at 2026-08-09 16:22:50 on dist-test-slave-s11t"
server_key: "03d46e5a03302c0a4e0834040ce4ca94"
server_key_iv: "9aa4f98d336e25d62e8c4e28112e6ba9"
server_key_version: "kuduclusterkey@0"
I20260809 16:22:50.776058 8982 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292417722232-21710-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292417722232-21710-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292417722232-21710-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:22:50.806236 8982 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:22:50.808988 8982 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:22:50.809168 8982 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:22:50.809433 8982 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:22:50.809870 8982 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:22:50.809929 8982 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:22:50.809963 8982 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:22:50.809990 8982 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:22:50.818482 8982 rpc_server.cc:307] RPC server started. Bound to: 127.21.51.131:42679
I20260809 16:22:50.818977 8982 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292417722232-21710-0/minicluster-data/ts-2/data/info.pb
I20260809 16:22:50.820137 21710 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrNqUCn/build/release/bin/kudu as pid 8982
I20260809 16:22:50.820266 21710 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292417722232-21710-0/minicluster-data/ts-2/wal/instance
I20260809 16:22:50.820462 21710 external_mini_cluster.cc:1428] Setting key 29fe4470291a062064221e2e26cee0be
Aug 09 16:22:50 dist-test-slave-s11t krb5kdc[7665](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292570, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.51.131@KRBTEST.COM for kudu/127.21.51.190@KRBTEST.COM
I20260809 16:22:50.819459 9182 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.51.131:42679 every 8 connection(s)
I20260809 16:22:50.832955 8774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:22:50.819817 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.51.131:33577 (local address 127.21.51.190:38333)
0809 16:22:50.820070 (+ 253us) server_negotiation.cc:206] Beginning negotiation
0809 16:22:50.820073 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 16:22:50.820908 (+ 835us) server_negotiation.cc:407] Connection header received
0809 16:22:50.821607 (+ 699us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:22:50.821609 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:22:50.821621 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:22:50.821689 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:22:50.822447 (+ 758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:22:50.823306 (+ 859us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:22:50.824928 (+ 1622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:22:50.825302 (+ 374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:22:50.825396 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:22:50.828420 (+ 3024us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:22:50.828422 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:22:50.828422 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:22:50.828430 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:22:50.830305 (+ 1875us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:22:50.830888 (+ 583us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:22:50.830889 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:22:50.830889 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:22:50.830960 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:22:50.831329 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:22:50.831330 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:22:50.831330 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:22:50.831484 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:22:50.831562 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0809 16:22:50.832755 (+ 1193us) server_negotiation.cc:299] Negotiation successful
0809 16:22:50.832851 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":219}
I20260809 16:22:50.833277 9185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:22:50.819796 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:38333 (local address 127.21.51.131:33577)
0809 16:22:50.820758 (+ 962us) negotiation.cc:107] Waiting for socket to connect
0809 16:22:50.820808 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0809 16:22:50.821483 (+ 675us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:22:50.821770 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:22:50.821775 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:22:50.822037 (+ 262us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:22:50.822328 (+ 291us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:22:50.822340 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:22:50.823424 (+ 1084us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:22:50.823425 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:22:50.824826 (+ 1401us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:22:50.824830 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:22:50.825448 (+ 618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:22:50.825449 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:22:50.825597 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:22:50.826299 (+ 702us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:22:50.826310 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:22:50.828300 (+ 1990us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:22:50.830398 (+ 2098us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:22:50.830407 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:22:50.830410 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:22:50.830789 (+ 379us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:22:50.831046 (+ 257us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:22:50.831047 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:22:50.831048 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:22:50.831228 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:22:50.832303 (+ 1075us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:22:50.832312 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:22:50.832531 (+ 219us) client_negotiation.cc:769] Sending connection context
0809 16:22:50.832739 (+ 208us) client_negotiation.cc:240] Negotiation successful
0809 16:22:50.832956 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":905,"thread_start_us":104,"threads_started":1}
I20260809 16:22:50.833864 9183 heartbeater.cc:347] Connected to a master server at 127.21.51.190:38333
I20260809 16:22:50.833979 9183 heartbeater.cc:464] Registering TS with master...
I20260809 16:22:50.834187 9183 heartbeater.cc:511] Master 127.21.51.190:38333 requested a full tablet report, sending...
I20260809 16:22:50.834601 8417 ts_manager.cc:194] Registered new tserver with Master: ef84aefd53974155b0d5ffa6d691b55f (127.21.51.131:42679)
I20260809 16:22:50.835108 8417 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.51.131@KRBTEST.COM'} at 127.21.51.131:33577
I20260809 16:22:50.843070 21710 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 16:22:50.843971 21710 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrNqUCn/build/release/bin/kudu with pid 8549
I20260809 16:22:50.859387 21710 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrNqUCn/build/release/bin/kudu with pid 8784
I20260809 16:22:50.868906 21710 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrNqUCn/build/release/bin/kudu with pid 8982
I20260809 16:22:50.877370 21710 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrNqUCn/build/release/bin/kudu with pid 8367
I20260809 16:22:50.888957 21710 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrNqUCn/build/release/bin/kudu
/tmp/dist-test-taskrNqUCn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292417722232-21710-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292417722232-21710-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292417722232-21710-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292417722232-21710-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.51.190:38333
--webserver_interface=127.21.51.190
--webserver_port=40711
--keytab_file=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.190.keytab
--principal=kudu/127.21.51.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.51.148:39109
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.51.190:38333
--ranger_config_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292417722232-21710-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.21.51.148:39707/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.21.51.148:39707/kms with env {KRB5CCNAME=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292417722232-21710-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292417722232-21710-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292417722232-21710-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:22:50.971858 9188 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:22:50.972060 9188 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:22:50.972083 9188 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:22:50.973238 9188 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 16:22:50.973279 9188 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:22:50.973294 9188 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:22:50.973306 9188 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 16:22:50.973318 9188 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 16:22:50.974968 9188 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.51.148:39109
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292417722232-21710-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292417722232-21710-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.21.51.148:39707/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.21.51.190:38333
--ranger_config_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292417722232-21710-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.190.keytab
--principal=kudu/127.21.51.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.51.190:38333
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292417722232-21710-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.51.190
--webserver_port=40711
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9188
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292417722232-21710-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:22:50.975227 9188 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:22:50.975463 9188 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:22:50.981392 9194 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:22:50.981417 9196 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:22:50.981391 9201 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:22:50.981724 9188 server_base.cc:1034] running on GCE node
I20260809 16:22:50.981962 9188 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:22:50.982247 9188 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:22:50.983435 9188 hybrid_clock.cc:648] HybridClock initialized: now 1786292570983423 us; error 31 us; skew 500 ppm
Aug 09 16:22:50 dist-test-slave-s11t krb5kdc[7665](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292570, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.51.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:22:50.986241 9188 init.cc:377] Logged in from keytab as kudu/127.21.51.190@KRBTEST.COM (short username kudu)
I20260809 16:22:50.986778 9188 webserver.cc:460] Webserver started at http://127.21.51.190:40711/ using document root <none> and password file <none>
I20260809 16:22:50.987035 9188 fs_manager.cc:359] Metadata directory not provided
I20260809 16:22:50.987115 9188 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 09 16:22:51 dist-test-slave-s11t krb5kdc[7665](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292570, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.51.190@KRBTEST.COM for HTTP/127.21.51.148@KRBTEST.COM
I20260809 16:22:51.033474 9188 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.001s sys 0.000s
I20260809 16:22:51.034401 9213 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:22:51.034612 9188 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:22:51.034714 9188 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292417722232-21710-0/minicluster-data/master-0/data,/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292417722232-21710-0/minicluster-data/master-0/wal
uuid: "860b05f8620c4cc2a5be9cec7d0e0fcb"
format_stamp: "Formatted at 2026-08-09 16:22:49 on dist-test-slave-s11t"
server_key: "3d1ed0c397b4eb09102098e541307065"
server_key_iv: "873f5335787222f984b3709845e19c08"
server_key_version: "kuduclusterkey@0"
I20260809 16:22:51.035162 9188 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292417722232-21710-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292417722232-21710-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292417722232-21710-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:22:51.057540 9188 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:22:51.059783 9188 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:22:51.059937 9188 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:22:51.065513 9188 rpc_server.cc:307] RPC server started. Bound to: 127.21.51.190:38333
I20260809 16:22:51.065550 9284 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.51.190:38333 every 8 connection(s)
I20260809 16:22:51.066007 9188 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292417722232-21710-0/minicluster-data/master-0/data/info.pb
I20260809 16:22:51.067351 9285 sys_catalog.cc:263] Verifying existing consensus state
I20260809 16:22:51.068049 9285 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 860b05f8620c4cc2a5be9cec7d0e0fcb: Bootstrap starting.
I20260809 16:22:51.075379 21710 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrNqUCn/build/release/bin/kudu as pid 9188
I20260809 16:22:51.075718 21710 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrNqUCn/build/release/bin/kudu
/tmp/dist-test-taskrNqUCn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292417722232-21710-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292417722232-21710-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292417722232-21710-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292417722232-21710-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.21.51.148:39707/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.51.129:44557
--local_ip_for_outbound_sockets=127.21.51.129
--tserver_master_addrs=127.21.51.190:38333
--webserver_port=37731
--webserver_interface=127.21.51.129
--keytab_file=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.129.keytab
--principal=kudu/127.21.51.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.51.148:39109
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.21.51.148:39707/kms with env {KRB5CCNAME=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292417722232-21710-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292417722232-21710-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292417722232-21710-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 16:22:51.077579 9285 log.cc:824] T 00000000000000000000000000000000 P 860b05f8620c4cc2a5be9cec7d0e0fcb: Log is configured to *not* fsync() on all Append() calls
I20260809 16:22:51.079632 9285 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 860b05f8620c4cc2a5be9cec7d0e0fcb: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 16:22:51.079947 9285 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 860b05f8620c4cc2a5be9cec7d0e0fcb: Bootstrap complete.
I20260809 16:22:51.082703 9285 raft_consensus.cc:348] T 00000000000000000000000000000000 P 860b05f8620c4cc2a5be9cec7d0e0fcb [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "860b05f8620c4cc2a5be9cec7d0e0fcb" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 38333 } }
I20260809 16:22:51.082973 9285 raft_consensus.cc:729] T 00000000000000000000000000000000 P 860b05f8620c4cc2a5be9cec7d0e0fcb [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 860b05f8620c4cc2a5be9cec7d0e0fcb, State: Initialized, Role: FOLLOWER
I20260809 16:22:51.083149 9285 consensus_queue.cc:260] T 00000000000000000000000000000000 P 860b05f8620c4cc2a5be9cec7d0e0fcb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "860b05f8620c4cc2a5be9cec7d0e0fcb" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 38333 } }
I20260809 16:22:51.083252 9285 raft_consensus.cc:388] T 00000000000000000000000000000000 P 860b05f8620c4cc2a5be9cec7d0e0fcb [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 16:22:51.083302 9285 raft_consensus.cc:482] T 00000000000000000000000000000000 P 860b05f8620c4cc2a5be9cec7d0e0fcb [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 16:22:51.083344 9285 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 860b05f8620c4cc2a5be9cec7d0e0fcb [term 1 FOLLOWER]: Advancing to term 2
I20260809 16:22:51.084342 9285 raft_consensus.cc:504] T 00000000000000000000000000000000 P 860b05f8620c4cc2a5be9cec7d0e0fcb [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "860b05f8620c4cc2a5be9cec7d0e0fcb" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 38333 } }
I20260809 16:22:51.084466 9285 leader_election.cc:302] T 00000000000000000000000000000000 P 860b05f8620c4cc2a5be9cec7d0e0fcb [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 860b05f8620c4cc2a5be9cec7d0e0fcb; no voters:
I20260809 16:22:51.084662 9285 leader_election.cc:288] T 00000000000000000000000000000000 P 860b05f8620c4cc2a5be9cec7d0e0fcb [CANDIDATE]: Term 2 election: Requested vote from peers
I20260809 16:22:51.084786 9290 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 860b05f8620c4cc2a5be9cec7d0e0fcb [term 2 FOLLOWER]: Leader election won for term 2
I20260809 16:22:51.084959 9290 raft_consensus.cc:686] T 00000000000000000000000000000000 P 860b05f8620c4cc2a5be9cec7d0e0fcb [term 2 LEADER]: Becoming Leader. State: Replica: 860b05f8620c4cc2a5be9cec7d0e0fcb, State: Running, Role: LEADER
I20260809 16:22:51.084997 9285 sys_catalog.cc:564] T 00000000000000000000000000000000 P 860b05f8620c4cc2a5be9cec7d0e0fcb [sys.catalog]: configured and running, proceeding with master startup.
I20260809 16:22:51.085680 9290 consensus_queue.cc:237] T 00000000000000000000000000000000 P 860b05f8620c4cc2a5be9cec7d0e0fcb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "860b05f8620c4cc2a5be9cec7d0e0fcb" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 38333 } }
I20260809 16:22:51.086086 9294 sys_catalog.cc:455] T 00000000000000000000000000000000 P 860b05f8620c4cc2a5be9cec7d0e0fcb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 860b05f8620c4cc2a5be9cec7d0e0fcb. Latest consensus state: current_term: 2 leader_uuid: "860b05f8620c4cc2a5be9cec7d0e0fcb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "860b05f8620c4cc2a5be9cec7d0e0fcb" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 38333 } } }
I20260809 16:22:51.086182 9294 sys_catalog.cc:458] T 00000000000000000000000000000000 P 860b05f8620c4cc2a5be9cec7d0e0fcb [sys.catalog]: This master's current role is: LEADER
I20260809 16:22:51.086522 9291 sys_catalog.cc:455] T 00000000000000000000000000000000 P 860b05f8620c4cc2a5be9cec7d0e0fcb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "860b05f8620c4cc2a5be9cec7d0e0fcb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "860b05f8620c4cc2a5be9cec7d0e0fcb" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 38333 } } }
I20260809 16:22:51.086591 9291 sys_catalog.cc:458] T 00000000000000000000000000000000 P 860b05f8620c4cc2a5be9cec7d0e0fcb [sys.catalog]: This master's current role is: LEADER
I20260809 16:22:51.086616 9285 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292417722232-21710-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260809 16:22:51.087247 9314 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 16:22:51.087733 9314 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 16:22:51.087896 9314 catalog_manager.cc:1259] Loaded cluster ID: 940f46a6dbb04cc9b0eaed9b671e5af3
I20260809 16:22:51.087937 9314 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 16:22:51.089128 9314 catalog_manager.cc:1504] Loading token signing keys...
I20260809 16:22:51.089316 9314 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 860b05f8620c4cc2a5be9cec7d0e0fcb: Loaded TSK: 0
I20260809 16:22:51.089550 9314 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260809 16:22:51.197122 9287 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:22:51.197405 9287 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:22:51.197458 9287 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:22:51.199146 9287 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:22:51.199214 9287 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:22:51.199261 9287 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.51.129
I20260809 16:22:51.201381 9287 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.51.148:39109
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292417722232-21710-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292417722232-21710-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.21.51.148:39707/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.129.keytab
--principal=kudu/127.21.51.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.51.129:44557
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292417722232-21710-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.51.129
--webserver_port=37731
--tserver_master_addrs=127.21.51.190:38333
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9287
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.51.129
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292417722232-21710-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:22:51.201635 9287 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:22:51.201923 9287 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:22:51.208355 9348 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:22:51.208385 9352 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:22:51.208778 9349 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:22:51.209515 9287 server_base.cc:1034] running on GCE node
I20260809 16:22:51.209733 9287 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:22:51.210021 9287 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:22:51.211210 9287 hybrid_clock.cc:648] HybridClock initialized: now 1786292571211179 us; error 45 us; skew 500 ppm
Aug 09 16:22:51 dist-test-slave-s11t krb5kdc[7665](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292571, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.51.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:22:51.214728 9287 init.cc:377] Logged in from keytab as kudu/127.21.51.129@KRBTEST.COM (short username kudu)
I20260809 16:22:51.215291 9287 webserver.cc:460] Webserver started at http://127.21.51.129:37731/ using document root <none> and password file <none>
I20260809 16:22:51.215592 9287 fs_manager.cc:359] Metadata directory not provided
I20260809 16:22:51.215680 9287 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 09 16:22:51 dist-test-slave-s11t krb5kdc[7665](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292571, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.51.129@KRBTEST.COM for HTTP/127.21.51.148@KRBTEST.COM
I20260809 16:22:51.246140 9287 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.000s sys 0.001s
I20260809 16:22:51.249023 9361 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:22:51.249274 9287 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.000s
I20260809 16:22:51.249362 9287 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292417722232-21710-0/minicluster-data/ts-0/data,/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292417722232-21710-0/minicluster-data/ts-0/wal
uuid: "22ecd31dbeeb44ad8fb4a150f0af7665"
format_stamp: "Formatted at 2026-08-09 16:22:50 on dist-test-slave-s11t"
server_key: "b3fb8b09c89cd591c01e85e1b18893f4"
server_key_iv: "aa1f3d19a8639f6daea85edfb0d3b58e"
server_key_version: "kuduclusterkey@0"
I20260809 16:22:51.249820 9287 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292417722232-21710-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292417722232-21710-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292417722232-21710-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:22:51.267943 9287 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:22:51.270493 9287 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:22:51.270658 9287 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:22:51.270879 9287 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:22:51.271277 9287 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:22:51.271350 9287 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:22:51.271400 9287 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:22:51.271436 9287 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:22:51.279088 9287 rpc_server.cc:307] RPC server started. Bound to: 127.21.51.129:44557
I20260809 16:22:51.279563 9287 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292417722232-21710-0/minicluster-data/ts-0/data/info.pb
I20260809 16:22:51.280843 9538 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.51.129:44557 every 8 connection(s)
Aug 09 16:22:51 dist-test-slave-s11t krb5kdc[7665](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292571, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.51.129@KRBTEST.COM for kudu/127.21.51.190@KRBTEST.COM
I20260809 16:22:51.288945 21710 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrNqUCn/build/release/bin/kudu as pid 9287
I20260809 16:22:51.289301 21710 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrNqUCn/build/release/bin/kudu
/tmp/dist-test-taskrNqUCn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292417722232-21710-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292417722232-21710-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292417722232-21710-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292417722232-21710-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.21.51.148:39707/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.51.130:43121
--local_ip_for_outbound_sockets=127.21.51.130
--tserver_master_addrs=127.21.51.190:38333
--webserver_port=43149
--webserver_interface=127.21.51.130
--keytab_file=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.130.keytab
--principal=kudu/127.21.51.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.51.148:39109
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.21.51.148:39707/kms with env {KRB5CCNAME=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292417722232-21710-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292417722232-21710-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292417722232-21710-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 16:22:51.292950 9544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:22:51.280430 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.51.129:50185 (local address 127.21.51.190:38333)
0809 16:22:51.280628 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0809 16:22:51.280637 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 16:22:51.281016 (+ 379us) server_negotiation.cc:407] Connection header received
0809 16:22:51.281860 (+ 844us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:22:51.281867 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:22:51.281930 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:22:51.282076 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:22:51.282807 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:22:51.283835 (+ 1028us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:22:51.285432 (+ 1597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:22:51.285825 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:22:51.285999 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:22:51.288625 (+ 2626us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:22:51.288630 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:22:51.288630 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:22:51.288642 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:22:51.290794 (+ 2152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:22:51.291325 (+ 531us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:22:51.291328 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:22:51.291328 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:22:51.291438 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:22:51.291756 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:22:51.291757 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:22:51.291757 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:22:51.292067 (+ 310us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:22:51.292134 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0809 16:22:51.292563 (+ 429us) server_negotiation.cc:299] Negotiation successful
0809 16:22:51.292727 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":150,"thread_start_us":79,"threads_started":1}
I20260809 16:22:51.293527 9542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:22:51.280357 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:38333 (local address 127.21.51.129:50185)
0809 16:22:51.280859 (+ 502us) negotiation.cc:107] Waiting for socket to connect
0809 16:22:51.280906 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0809 16:22:51.281610 (+ 704us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:22:51.282139 (+ 529us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:22:51.282143 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:22:51.282381 (+ 238us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:22:51.282657 (+ 276us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:22:51.282663 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:22:51.283954 (+ 1291us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:22:51.283955 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:22:51.285335 (+ 1380us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:22:51.285338 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:22:51.286013 (+ 675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:22:51.286014 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:22:51.286183 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:22:51.286824 (+ 641us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:22:51.286833 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:22:51.288504 (+ 1671us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:22:51.290895 (+ 2391us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:22:51.290901 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:22:51.290903 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:22:51.291204 (+ 301us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:22:51.291527 (+ 323us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:22:51.291528 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:22:51.291528 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:22:51.291668 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:22:51.292190 (+ 522us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:22:51.292196 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:22:51.292359 (+ 163us) client_negotiation.cc:769] Sending connection context
0809 16:22:51.293141 (+ 782us) client_negotiation.cc:240] Negotiation successful
0809 16:22:51.293315 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":458,"thread_start_us":89,"threads_started":1}
I20260809 16:22:51.294600 9539 heartbeater.cc:347] Connected to a master server at 127.21.51.190:38333
I20260809 16:22:51.294692 9539 heartbeater.cc:464] Registering TS with master...
I20260809 16:22:51.294873 9539 heartbeater.cc:511] Master 127.21.51.190:38333 requested a full tablet report, sending...
W20260809 16:22:51.295810 9539 heartbeater.cc:644] Failed to heartbeat to 127.21.51.190:38333 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260809 16:22:51.295878 9539 heartbeater.cc:464] Registering TS with master...
I20260809 16:22:51.295941 9539 heartbeater.cc:511] Master 127.21.51.190:38333 requested a full tablet report, sending...
I20260809 16:22:51.296336 9539 heartbeater.cc:464] Registering TS with master...
I20260809 16:22:51.296422 9539 heartbeater.cc:511] Master 127.21.51.190:38333 requested a full tablet report, sending...
W20260809 16:22:51.296806 9539 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260809 16:22:51.401722 9547 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:22:51.402066 9547 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:22:51.402153 9547 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:22:51.403676 9547 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:22:51.403791 9547 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:22:51.403901 9547 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.51.130
I20260809 16:22:51.406011 9547 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.51.148:39109
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292417722232-21710-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292417722232-21710-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.21.51.148:39707/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.130.keytab
--principal=kudu/127.21.51.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.51.130:43121
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292417722232-21710-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.51.130
--webserver_port=43149
--tserver_master_addrs=127.21.51.190:38333
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9547
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.51.130
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292417722232-21710-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:22:51.406284 9547 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:22:51.406531 9547 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:22:51.412703 9553 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:22:51.412930 9547 server_base.cc:1034] running on GCE node
W20260809 16:22:51.416846 9554 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:22:51.417963 9557 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:22:51.418249 9547 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:22:51.418533 9547 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:22:51.419739 9547 hybrid_clock.cc:648] HybridClock initialized: now 1786292571419712 us; error 41 us; skew 500 ppm
Aug 09 16:22:51 dist-test-slave-s11t krb5kdc[7665](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292571, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.51.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:22:51.422479 9547 init.cc:377] Logged in from keytab as kudu/127.21.51.130@KRBTEST.COM (short username kudu)
I20260809 16:22:51.422994 9547 webserver.cc:460] Webserver started at http://127.21.51.130:43149/ using document root <none> and password file <none>
I20260809 16:22:51.423249 9547 fs_manager.cc:359] Metadata directory not provided
I20260809 16:22:51.423336 9547 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 09 16:22:51 dist-test-slave-s11t krb5kdc[7665](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292571, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.51.130@KRBTEST.COM for HTTP/127.21.51.148@KRBTEST.COM
I20260809 16:22:51.442299 9547 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:22:51.444224 9571 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:22:51.444402 9547 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.000s
I20260809 16:22:51.444484 9547 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292417722232-21710-0/minicluster-data/ts-1/data,/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292417722232-21710-0/minicluster-data/ts-1/wal
uuid: "06717649045b4dc685420de2d8faec74"
format_stamp: "Formatted at 2026-08-09 16:22:50 on dist-test-slave-s11t"
server_key: "dfa6a5c5fb641b9aab8570dfbfd24422"
server_key_iv: "ad7492344a479b25a12bba94c1b5a492"
server_key_version: "kuduclusterkey@0"
I20260809 16:22:51.444978 9547 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292417722232-21710-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292417722232-21710-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292417722232-21710-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:22:51.458035 9547 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:22:51.460578 9547 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:22:51.460807 9547 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:22:51.461031 9547 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:22:51.461511 9547 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:22:51.461570 9547 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:22:51.461601 9547 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:22:51.461619 9547 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:22:51.471490 9547 rpc_server.cc:307] RPC server started. Bound to: 127.21.51.130:43121
I20260809 16:22:51.472100 9547 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292417722232-21710-0/minicluster-data/ts-1/data/info.pb
I20260809 16:22:51.473908 21710 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrNqUCn/build/release/bin/kudu as pid 9547
I20260809 16:22:51.474269 21710 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrNqUCn/build/release/bin/kudu
/tmp/dist-test-taskrNqUCn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292417722232-21710-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292417722232-21710-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292417722232-21710-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292417722232-21710-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.21.51.148:39707/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.51.131:42679
--local_ip_for_outbound_sockets=127.21.51.131
--tserver_master_addrs=127.21.51.190:38333
--webserver_port=34663
--webserver_interface=127.21.51.131
--keytab_file=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.131.keytab
--principal=kudu/127.21.51.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.51.148:39109
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.21.51.148:39707/kms with env {KRB5CCNAME=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292417722232-21710-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292417722232-21710-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292417722232-21710-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 16:22:51.474712 9755 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.51.130:43121 every 8 connection(s)
Aug 09 16:22:51 dist-test-slave-s11t krb5kdc[7665](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292571, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.51.130@KRBTEST.COM for kudu/127.21.51.190@KRBTEST.COM
I20260809 16:22:51.487545 9544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:22:51.475486 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.51.130:44119 (local address 127.21.51.190:38333)
0809 16:22:51.475548 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0809 16:22:51.475550 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:22:51.475803 (+ 253us) server_negotiation.cc:407] Connection header received
0809 16:22:51.476489 (+ 686us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:22:51.476491 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:22:51.476505 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:22:51.476535 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:22:51.477401 (+ 866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:22:51.478188 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:22:51.479734 (+ 1546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:22:51.480052 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:22:51.480099 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:22:51.484063 (+ 3964us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:22:51.484065 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:22:51.484065 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:22:51.484073 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:22:51.485842 (+ 1769us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:22:51.486385 (+ 543us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:22:51.486386 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:22:51.486386 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:22:51.486452 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:22:51.486755 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:22:51.486756 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:22:51.486756 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:22:51.486907 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:22:51.486983 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0809 16:22:51.487353 (+ 370us) server_negotiation.cc:299] Negotiation successful
0809 16:22:51.487443 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260809 16:22:51.487865 9759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:22:51.475494 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:38333 (local address 127.21.51.130:44119)
0809 16:22:51.475662 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0809 16:22:51.475704 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0809 16:22:51.476354 (+ 650us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:22:51.476754 (+ 400us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:22:51.476758 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:22:51.476993 (+ 235us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:22:51.477280 (+ 287us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:22:51.477286 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:22:51.478316 (+ 1030us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:22:51.478317 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:22:51.479624 (+ 1307us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:22:51.479627 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:22:51.480223 (+ 596us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:22:51.480224 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:22:51.480374 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:22:51.482231 (+ 1857us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:22:51.482242 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:22:51.483944 (+ 1702us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:22:51.485943 (+ 1999us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:22:51.485950 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:22:51.485953 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:22:51.486294 (+ 341us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:22:51.486519 (+ 225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:22:51.486519 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:22:51.486520 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:22:51.486691 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:22:51.486990 (+ 299us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:22:51.486996 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:22:51.487169 (+ 173us) client_negotiation.cc:769] Sending connection context
0809 16:22:51.487308 (+ 139us) client_negotiation.cc:240] Negotiation successful
0809 16:22:51.487480 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":99,"threads_started":1}
I20260809 16:22:51.488519 9757 heartbeater.cc:347] Connected to a master server at 127.21.51.190:38333
I20260809 16:22:51.488628 9757 heartbeater.cc:464] Registering TS with master...
I20260809 16:22:51.488868 9757 heartbeater.cc:511] Master 127.21.51.190:38333 requested a full tablet report, sending...
W20260809 16:22:51.489641 9757 heartbeater.cc:644] Failed to heartbeat to 127.21.51.190:38333 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260809 16:22:51.489717 9757 heartbeater.cc:464] Registering TS with master...
I20260809 16:22:51.489777 9757 heartbeater.cc:511] Master 127.21.51.190:38333 requested a full tablet report, sending...
I20260809 16:22:51.490259 9757 heartbeater.cc:464] Registering TS with master...
I20260809 16:22:51.490347 9757 heartbeater.cc:511] Master 127.21.51.190:38333 requested a full tablet report, sending...
W20260809 16:22:51.490676 9757 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260809 16:22:51.612665 9760 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:22:51.612996 9760 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:22:51.613039 9760 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:22:51.614745 9760 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:22:51.614806 9760 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:22:51.614848 9760 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.51.131
I20260809 16:22:51.616844 9760 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.51.148:39109
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292417722232-21710-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292417722232-21710-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.21.51.148:39707/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.131.keytab
--principal=kudu/127.21.51.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.51.131:42679
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292417722232-21710-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.51.131
--webserver_port=34663
--tserver_master_addrs=127.21.51.190:38333
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9760
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.51.131
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292417722232-21710-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:22:51.617105 9760 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:22:51.617362 9760 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:22:51.623608 9770 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:22:51.624441 9771 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:22:51.624637 9774 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:22:51.625109 9760 server_base.cc:1034] running on GCE node
I20260809 16:22:51.625331 9760 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:22:51.625586 9760 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:22:51.626747 9760 hybrid_clock.cc:648] HybridClock initialized: now 1786292571626713 us; error 48 us; skew 500 ppm
Aug 09 16:22:51 dist-test-slave-s11t krb5kdc[7665](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292571, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.51.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:22:51.630244 9760 init.cc:377] Logged in from keytab as kudu/127.21.51.131@KRBTEST.COM (short username kudu)
I20260809 16:22:51.630762 9760 webserver.cc:460] Webserver started at http://127.21.51.131:34663/ using document root <none> and password file <none>
I20260809 16:22:51.631084 9760 fs_manager.cc:359] Metadata directory not provided
I20260809 16:22:51.631184 9760 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 09 16:22:51 dist-test-slave-s11t krb5kdc[7665](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292571, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.51.131@KRBTEST.COM for HTTP/127.21.51.148@KRBTEST.COM
I20260809 16:22:51.647571 9760 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:22:51.648644 9785 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:22:51.648854 9760 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:22:51.648957 9760 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292417722232-21710-0/minicluster-data/ts-2/data,/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292417722232-21710-0/minicluster-data/ts-2/wal
uuid: "ef84aefd53974155b0d5ffa6d691b55f"
format_stamp: "Formatted at 2026-08-09 16:22:50 on dist-test-slave-s11t"
server_key: "03d46e5a03302c0a4e0834040ce4ca94"
server_key_iv: "9aa4f98d336e25d62e8c4e28112e6ba9"
server_key_version: "kuduclusterkey@0"
I20260809 16:22:51.649477 9760 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292417722232-21710-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292417722232-21710-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292417722232-21710-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:22:51.661203 9760 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:22:51.663439 9760 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:22:51.663640 9760 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:22:51.663867 9760 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:22:51.664309 9760 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:22:51.664368 9760 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:22:51.664412 9760 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:22:51.664439 9760 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:22:51.672072 9760 rpc_server.cc:307] RPC server started. Bound to: 127.21.51.131:42679
I20260809 16:22:51.672657 9760 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292417722232-21710-0/minicluster-data/ts-2/data/info.pb
I20260809 16:22:51.673326 9967 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.51.131:42679 every 8 connection(s)
I20260809 16:22:51.678987 21710 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrNqUCn/build/release/bin/kudu as pid 9760
Aug 09 16:22:51 dist-test-slave-s11t krb5kdc[7665](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292571, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.51.131@KRBTEST.COM for kudu/127.21.51.190@KRBTEST.COM
I20260809 16:22:51.684257 9544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:22:51.674057 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.51.131:32819 (local address 127.21.51.190:38333)
0809 16:22:51.674136 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0809 16:22:51.674138 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:22:51.674353 (+ 215us) server_negotiation.cc:407] Connection header received
0809 16:22:51.674937 (+ 584us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:22:51.674939 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:22:51.674953 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:22:51.674988 (+ 35us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:22:51.675853 (+ 865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:22:51.676646 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:22:51.678131 (+ 1485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:22:51.678353 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:22:51.678451 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:22:51.680909 (+ 2458us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:22:51.680910 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:22:51.680910 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:22:51.680918 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:22:51.682758 (+ 1840us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:22:51.683164 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:22:51.683166 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:22:51.683166 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:22:51.683234 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:22:51.683502 (+ 268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:22:51.683504 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:22:51.683504 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:22:51.683620 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:22:51.683706 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0809 16:22:51.684082 (+ 376us) server_negotiation.cc:299] Negotiation successful
0809 16:22:51.684165 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260809 16:22:51.684404 9974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:22:51.674053 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:38333 (local address 127.21.51.131:32819)
0809 16:22:51.674218 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0809 16:22:51.674260 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0809 16:22:51.674797 (+ 537us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:22:51.675182 (+ 385us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:22:51.675187 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:22:51.675437 (+ 250us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:22:51.675721 (+ 284us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:22:51.675728 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:22:51.676811 (+ 1083us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:22:51.676812 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:22:51.678031 (+ 1219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:22:51.678033 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:22:51.678481 (+ 448us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:22:51.678483 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:22:51.678611 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:22:51.679168 (+ 557us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:22:51.679177 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:22:51.680801 (+ 1624us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:22:51.682850 (+ 2049us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:22:51.682856 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:22:51.682857 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:22:51.683070 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:22:51.683325 (+ 255us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:22:51.683326 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:22:51.683326 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:22:51.683425 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:22:51.683736 (+ 311us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:22:51.683742 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:22:51.683909 (+ 167us) client_negotiation.cc:769] Sending connection context
0809 16:22:51.684027 (+ 118us) client_negotiation.cc:240] Negotiation successful
0809 16:22:51.684178 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":97,"threads_started":1}
I20260809 16:22:51.685032 9969 heartbeater.cc:347] Connected to a master server at 127.21.51.190:38333
I20260809 16:22:51.685146 9969 heartbeater.cc:464] Registering TS with master...
I20260809 16:22:51.685402 9969 heartbeater.cc:511] Master 127.21.51.190:38333 requested a full tablet report, sending...
W20260809 16:22:51.686167 9969 heartbeater.cc:644] Failed to heartbeat to 127.21.51.190:38333 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260809 16:22:51.686264 9969 heartbeater.cc:464] Registering TS with master...
I20260809 16:22:51.686339 9969 heartbeater.cc:511] Master 127.21.51.190:38333 requested a full tablet report, sending...
I20260809 16:22:51.686657 9969 heartbeater.cc:464] Registering TS with master...
I20260809 16:22:51.686734 9969 heartbeater.cc:511] Master 127.21.51.190:38333 requested a full tablet report, sending...
W20260809 16:22:51.687153 9969 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260809 16:22:51.692899 9975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:22:51.679410 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39742 (local address 127.21.51.190:38333)
0809 16:22:51.680774 (+ 1364us) server_negotiation.cc:206] Beginning negotiation
0809 16:22:51.680776 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:22:51.681184 (+ 408us) server_negotiation.cc:407] Connection header received
0809 16:22:51.681208 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:22:51.681209 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:22:51.681223 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:22:51.681310 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:22:51.682038 (+ 728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:22:51.683579 (+ 1541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:22:51.684898 (+ 1319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:22:51.685185 (+ 287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:22:51.685245 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:22:51.686791 (+ 1546us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:22:51.686793 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:22:51.686793 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:22:51.686800 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:22:51.688384 (+ 1584us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:22:51.692012 (+ 3628us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:22:51.692013 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:22:51.692013 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:22:51.692068 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:22:51.692296 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:22:51.692297 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:22:51.692298 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:22:51.692434 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:22:51.692616 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0809 16:22:51.692737 (+ 121us) server_negotiation.cc:299] Negotiation successful
0809 16:22:51.692826 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1332,"thread_start_us":78,"threads_started":1}
Aug 09 16:22:52 dist-test-slave-s11t krb5kdc[7665](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786292572, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.51.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:22:52.296994 9539 heartbeater.cc:464] Registering TS with master...
I20260809 16:22:52.297151 9539 heartbeater.cc:511] Master 127.21.51.190:38333 requested a full tablet report, sending...
I20260809 16:22:52.297580 9237 ts_manager.cc:194] Registered new tserver with Master: 22ecd31dbeeb44ad8fb4a150f0af7665 (127.21.51.129:44557)
I20260809 16:22:52.298604 9237 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.51.129@KRBTEST.COM'} at 127.21.51.129:50185
I20260809 16:22:52.490895 9757 heartbeater.cc:464] Registering TS with master...
I20260809 16:22:52.491060 9757 heartbeater.cc:511] Master 127.21.51.190:38333 requested a full tablet report, sending...
I20260809 16:22:52.491416 9237 ts_manager.cc:194] Registered new tserver with Master: 06717649045b4dc685420de2d8faec74 (127.21.51.130:43121)
I20260809 16:22:52.491981 9237 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.51.130@KRBTEST.COM'} at 127.21.51.130:44119
Aug 09 16:22:52 dist-test-slave-s11t krb5kdc[7665](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292572, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.51.190@KRBTEST.COM for HTTP/127.21.51.148@KRBTEST.COM
I20260809 16:22:52.687361 9969 heartbeater.cc:464] Registering TS with master...
I20260809 16:22:52.687534 9969 heartbeater.cc:511] Master 127.21.51.190:38333 requested a full tablet report, sending...
I20260809 16:22:52.687863 9237 ts_manager.cc:194] Registered new tserver with Master: ef84aefd53974155b0d5ffa6d691b55f (127.21.51.131:42679)
I20260809 16:22:52.688400 9237 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.51.131@KRBTEST.COM'} at 127.21.51.131:32819
I20260809 16:22:52.693601 21710 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 16:22:52.714030 10001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:22:52.697387 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39744 (local address 127.21.51.190:38333)
0809 16:22:52.704795 (+ 7408us) server_negotiation.cc:206] Beginning negotiation
0809 16:22:52.704797 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:22:52.704808 (+ 11us) server_negotiation.cc:407] Connection header received
0809 16:22:52.704834 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:22:52.704835 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:22:52.704848 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:22:52.705690 (+ 842us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:22:52.705732 (+ 42us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:22:52.706504 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:22:52.707563 (+ 1059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:22:52.707865 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:22:52.710447 (+ 2582us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:22:52.710500 (+ 53us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:22:52.710501 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:22:52.710501 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:22:52.710506 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:22:52.712395 (+ 1889us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:22:52.712830 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:22:52.712831 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:22:52.712831 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:22:52.712909 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:22:52.713135 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:22:52.713136 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:22:52.713137 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:22:52.713268 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:22:52.713779 (+ 511us) server_negotiation.cc:1035] Waiting for connection context
0809 16:22:52.713822 (+ 43us) server_negotiation.cc:299] Negotiation successful
0809 16:22:52.713918 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7382,"thread_start_us":84,"threads_started":1}
I20260809 16:22:52.716468 9237 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39744:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 16:22:52.718887 9237 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 16:22:52.733686 10017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:22:52.727420 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.130:43121 (local address 127.0.0.1:46664)
0809 16:22:52.728503 (+ 1083us) negotiation.cc:107] Waiting for socket to connect
0809 16:22:52.728512 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 16:22:52.728560 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:22:52.728882 (+ 322us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:22:52.728886 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:22:52.728888 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:22:52.728952 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:22:52.728954 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:22:52.730592 (+ 1638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:22:52.730593 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:22:52.731870 (+ 1277us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:22:52.731886 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:22:52.733366 (+ 1480us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:22:52.733367 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:22:52.733455 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:22:52.733470 (+ 15us) client_negotiation.cc:769] Sending connection context
0809 16:22:52.733546 (+ 76us) client_negotiation.cc:240] Negotiation successful
0809 16:22:52.733601 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1069,"thread_start_us":63,"threads_started":1}
I20260809 16:22:52.734167 10019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:22:52.727555 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46664 (local address 127.21.51.130:43121)
0809 16:22:52.728450 (+ 895us) server_negotiation.cc:206] Beginning negotiation
0809 16:22:52.728454 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 16:22:52.728611 (+ 157us) server_negotiation.cc:407] Connection header received
0809 16:22:52.728646 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:22:52.728648 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:22:52.728663 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:22:52.728770 (+ 107us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:22:52.729032 (+ 262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:22:52.730489 (+ 1457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:22:52.732024 (+ 1535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:22:52.733182 (+ 1158us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:22:52.733264 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:22:52.733324 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0809 16:22:52.734039 (+ 715us) server_negotiation.cc:299] Negotiation successful
0809 16:22:52.734085 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":864,"thread_start_us":55,"threads_started":1}
I20260809 16:22:52.735168 9649 tablet_service.cc:1467] Processing CreateTablet for tablet 67edd40372404e438b0d982421c2f163 (DEFAULT_TABLE table=test-table [id=0df183dee8a14068a91057148733ef1f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:22:52.735530 9649 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 67edd40372404e438b0d982421c2f163. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:22:52.739550 10026 tablet_bootstrap.cc:492] T 67edd40372404e438b0d982421c2f163 P 06717649045b4dc685420de2d8faec74: Bootstrap starting.
I20260809 16:22:52.740453 10026 tablet_bootstrap.cc:654] T 67edd40372404e438b0d982421c2f163 P 06717649045b4dc685420de2d8faec74: Neither blocks nor log segments found. Creating new log.
I20260809 16:22:52.740937 10026 log.cc:824] T 67edd40372404e438b0d982421c2f163 P 06717649045b4dc685420de2d8faec74: Log is configured to *not* fsync() on all Append() calls
I20260809 16:22:52.741139 10021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:22:52.727216 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.131:42679 (local address 127.0.0.1:43092)
0809 16:22:52.728211 (+ 995us) negotiation.cc:107] Waiting for socket to connect
0809 16:22:52.728255 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 16:22:52.728322 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:22:52.735463 (+ 7141us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:22:52.735464 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:22:52.735466 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:22:52.735507 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:22:52.735508 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:22:52.738715 (+ 3207us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:22:52.738716 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:22:52.739861 (+ 1145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:22:52.739864 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:22:52.740922 (+ 1058us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:22:52.740924 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:22:52.740990 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:22:52.740994 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 16:22:52.741026 (+ 32us) client_negotiation.cc:240] Negotiation successful
0809 16:22:52.741064 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":964,"thread_start_us":70,"threads_started":1}
I20260809 16:22:52.741490 10016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:22:52.727314 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43092 (local address 127.21.51.131:42679)
0809 16:22:52.735291 (+ 7977us) server_negotiation.cc:206] Beginning negotiation
0809 16:22:52.735296 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 16:22:52.735308 (+ 12us) server_negotiation.cc:407] Connection header received
0809 16:22:52.735343 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:22:52.735345 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:22:52.735361 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:22:52.735572 (+ 211us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:22:52.735661 (+ 89us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:22:52.736678 (+ 1017us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:22:52.739986 (+ 3308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:22:52.740800 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:22:52.741264 (+ 464us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:22:52.741341 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0809 16:22:52.741368 (+ 27us) server_negotiation.cc:299] Negotiation successful
0809 16:22:52.741411 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7937,"thread_start_us":69,"threads_started":1}
I20260809 16:22:52.742208 9834 tablet_service.cc:1467] Processing CreateTablet for tablet 67edd40372404e438b0d982421c2f163 (DEFAULT_TABLE table=test-table [id=0df183dee8a14068a91057148733ef1f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:22:52.742478 10026 tablet_bootstrap.cc:492] T 67edd40372404e438b0d982421c2f163 P 06717649045b4dc685420de2d8faec74: No bootstrap required, opened a new log
I20260809 16:22:52.742542 10026 ts_tablet_manager.cc:1397] T 67edd40372404e438b0d982421c2f163 P 06717649045b4dc685420de2d8faec74: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260809 16:22:52.742563 9834 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 67edd40372404e438b0d982421c2f163. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:22:52.745340 10026 raft_consensus.cc:348] T 67edd40372404e438b0d982421c2f163 P 06717649045b4dc685420de2d8faec74 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22ecd31dbeeb44ad8fb4a150f0af7665" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 44557 } } peers { permanent_uuid: "ef84aefd53974155b0d5ffa6d691b55f" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 42679 } } peers { permanent_uuid: "06717649045b4dc685420de2d8faec74" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 43121 } }
I20260809 16:22:52.745602 10026 raft_consensus.cc:374] T 67edd40372404e438b0d982421c2f163 P 06717649045b4dc685420de2d8faec74 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:22:52.745723 10026 raft_consensus.cc:729] T 67edd40372404e438b0d982421c2f163 P 06717649045b4dc685420de2d8faec74 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 06717649045b4dc685420de2d8faec74, State: Initialized, Role: FOLLOWER
I20260809 16:22:52.745924 10017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:22:52.727769 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.129:44557 (local address 127.0.0.1:52318)
0809 16:22:52.734739 (+ 6970us) negotiation.cc:107] Waiting for socket to connect
0809 16:22:52.734751 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 16:22:52.734795 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:22:52.741067 (+ 6272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:22:52.741068 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:22:52.741070 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:22:52.741108 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:22:52.741109 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:22:52.742415 (+ 1306us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:22:52.742416 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:22:52.744728 (+ 2312us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:22:52.744731 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:22:52.745701 (+ 970us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:22:52.745703 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:22:52.745767 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:22:52.745772 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 16:22:52.745808 (+ 36us) client_negotiation.cc:240] Negotiation successful
0809 16:22:52.745849 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6959,"thread_start_us":62,"threads_started":1}
I20260809 16:22:52.745936 10026 consensus_queue.cc:260] T 67edd40372404e438b0d982421c2f163 P 06717649045b4dc685420de2d8faec74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22ecd31dbeeb44ad8fb4a150f0af7665" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 44557 } } peers { permanent_uuid: "ef84aefd53974155b0d5ffa6d691b55f" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 42679 } } peers { permanent_uuid: "06717649045b4dc685420de2d8faec74" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 43121 } }
I20260809 16:22:52.746474 10026 ts_tablet_manager.cc:1428] T 67edd40372404e438b0d982421c2f163 P 06717649045b4dc685420de2d8faec74: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260809 16:22:52.746486 10024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:22:52.727861 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52318 (local address 127.21.51.129:44557)
0809 16:22:52.740791 (+ 12930us) server_negotiation.cc:206] Beginning negotiation
0809 16:22:52.740796 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 16:22:52.740808 (+ 12us) server_negotiation.cc:407] Connection header received
0809 16:22:52.740846 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:22:52.740848 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:22:52.740865 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:22:52.740958 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:22:52.741195 (+ 237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:22:52.742300 (+ 1105us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:22:52.744840 (+ 2540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:22:52.745595 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:22:52.746267 (+ 672us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:22:52.746338 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0809 16:22:52.746368 (+ 30us) server_negotiation.cc:299] Negotiation successful
0809 16:22:52.746411 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12883,"thread_start_us":50,"threads_started":1}
I20260809 16:22:52.746749 9757 heartbeater.cc:503] Master 127.21.51.190:38333 was elected leader, sending a full tablet report...
I20260809 16:22:52.747071 9431 tablet_service.cc:1467] Processing CreateTablet for tablet 67edd40372404e438b0d982421c2f163 (DEFAULT_TABLE table=test-table [id=0df183dee8a14068a91057148733ef1f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:22:52.747417 9431 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 67edd40372404e438b0d982421c2f163. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:22:52.748193 10031 tablet_bootstrap.cc:492] T 67edd40372404e438b0d982421c2f163 P ef84aefd53974155b0d5ffa6d691b55f: Bootstrap starting.
I20260809 16:22:52.748917 10031 tablet_bootstrap.cc:654] T 67edd40372404e438b0d982421c2f163 P ef84aefd53974155b0d5ffa6d691b55f: Neither blocks nor log segments found. Creating new log.
I20260809 16:22:52.749392 10031 log.cc:824] T 67edd40372404e438b0d982421c2f163 P ef84aefd53974155b0d5ffa6d691b55f: Log is configured to *not* fsync() on all Append() calls
I20260809 16:22:52.750332 10031 tablet_bootstrap.cc:492] T 67edd40372404e438b0d982421c2f163 P ef84aefd53974155b0d5ffa6d691b55f: No bootstrap required, opened a new log
I20260809 16:22:52.750396 10031 ts_tablet_manager.cc:1397] T 67edd40372404e438b0d982421c2f163 P ef84aefd53974155b0d5ffa6d691b55f: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 16:22:52.753001 10031 raft_consensus.cc:348] T 67edd40372404e438b0d982421c2f163 P ef84aefd53974155b0d5ffa6d691b55f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22ecd31dbeeb44ad8fb4a150f0af7665" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 44557 } } peers { permanent_uuid: "ef84aefd53974155b0d5ffa6d691b55f" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 42679 } } peers { permanent_uuid: "06717649045b4dc685420de2d8faec74" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 43121 } }
I20260809 16:22:52.753223 10031 raft_consensus.cc:374] T 67edd40372404e438b0d982421c2f163 P ef84aefd53974155b0d5ffa6d691b55f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:22:52.753296 10031 raft_consensus.cc:729] T 67edd40372404e438b0d982421c2f163 P ef84aefd53974155b0d5ffa6d691b55f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef84aefd53974155b0d5ffa6d691b55f, State: Initialized, Role: FOLLOWER
I20260809 16:22:52.753517 10031 consensus_queue.cc:260] T 67edd40372404e438b0d982421c2f163 P ef84aefd53974155b0d5ffa6d691b55f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22ecd31dbeeb44ad8fb4a150f0af7665" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 44557 } } peers { permanent_uuid: "ef84aefd53974155b0d5ffa6d691b55f" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 42679 } } peers { permanent_uuid: "06717649045b4dc685420de2d8faec74" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 43121 } }
I20260809 16:22:52.753815 10031 ts_tablet_manager.cc:1428] T 67edd40372404e438b0d982421c2f163 P ef84aefd53974155b0d5ffa6d691b55f: Time spent starting tablet: real 0.003s user 0.001s sys 0.002s
I20260809 16:22:52.753957 9969 heartbeater.cc:503] Master 127.21.51.190:38333 was elected leader, sending a full tablet report...
I20260809 16:22:52.754760 10036 tablet_bootstrap.cc:492] T 67edd40372404e438b0d982421c2f163 P 22ecd31dbeeb44ad8fb4a150f0af7665: Bootstrap starting.
I20260809 16:22:52.755410 10036 tablet_bootstrap.cc:654] T 67edd40372404e438b0d982421c2f163 P 22ecd31dbeeb44ad8fb4a150f0af7665: Neither blocks nor log segments found. Creating new log.
I20260809 16:22:52.755760 10036 log.cc:824] T 67edd40372404e438b0d982421c2f163 P 22ecd31dbeeb44ad8fb4a150f0af7665: Log is configured to *not* fsync() on all Append() calls
I20260809 16:22:52.756939 10036 tablet_bootstrap.cc:492] T 67edd40372404e438b0d982421c2f163 P 22ecd31dbeeb44ad8fb4a150f0af7665: No bootstrap required, opened a new log
I20260809 16:22:52.757000 10036 ts_tablet_manager.cc:1397] T 67edd40372404e438b0d982421c2f163 P 22ecd31dbeeb44ad8fb4a150f0af7665: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 16:22:52.759320 10036 raft_consensus.cc:348] T 67edd40372404e438b0d982421c2f163 P 22ecd31dbeeb44ad8fb4a150f0af7665 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22ecd31dbeeb44ad8fb4a150f0af7665" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 44557 } } peers { permanent_uuid: "ef84aefd53974155b0d5ffa6d691b55f" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 42679 } } peers { permanent_uuid: "06717649045b4dc685420de2d8faec74" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 43121 } }
I20260809 16:22:52.759461 10036 raft_consensus.cc:374] T 67edd40372404e438b0d982421c2f163 P 22ecd31dbeeb44ad8fb4a150f0af7665 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:22:52.759490 10036 raft_consensus.cc:729] T 67edd40372404e438b0d982421c2f163 P 22ecd31dbeeb44ad8fb4a150f0af7665 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 22ecd31dbeeb44ad8fb4a150f0af7665, State: Initialized, Role: FOLLOWER
I20260809 16:22:52.759590 10036 consensus_queue.cc:260] T 67edd40372404e438b0d982421c2f163 P 22ecd31dbeeb44ad8fb4a150f0af7665 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22ecd31dbeeb44ad8fb4a150f0af7665" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 44557 } } peers { permanent_uuid: "ef84aefd53974155b0d5ffa6d691b55f" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 42679 } } peers { permanent_uuid: "06717649045b4dc685420de2d8faec74" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 43121 } }
I20260809 16:22:52.759824 10036 ts_tablet_manager.cc:1428] T 67edd40372404e438b0d982421c2f163 P 22ecd31dbeeb44ad8fb4a150f0af7665: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 16:22:52.760237 9539 heartbeater.cc:503] Master 127.21.51.190:38333 was elected leader, sending a full tablet report...
W20260809 16:22:52.781111 9541 tablet.cc:2369] T 67edd40372404e438b0d982421c2f163 P 22ecd31dbeeb44ad8fb4a150f0af7665: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 16:22:52.862999 10040 raft_consensus.cc:482] T 67edd40372404e438b0d982421c2f163 P 22ecd31dbeeb44ad8fb4a150f0af7665 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 16:22:52.863144 10040 raft_consensus.cc:504] T 67edd40372404e438b0d982421c2f163 P 22ecd31dbeeb44ad8fb4a150f0af7665 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22ecd31dbeeb44ad8fb4a150f0af7665" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 44557 } } peers { permanent_uuid: "ef84aefd53974155b0d5ffa6d691b55f" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 42679 } } peers { permanent_uuid: "06717649045b4dc685420de2d8faec74" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 43121 } }
I20260809 16:22:52.863549 10040 leader_election.cc:288] T 67edd40372404e438b0d982421c2f163 P 22ecd31dbeeb44ad8fb4a150f0af7665 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ef84aefd53974155b0d5ffa6d691b55f (127.21.51.131:42679), 06717649045b4dc685420de2d8faec74 (127.21.51.130:43121)
Aug 09 16:22:52 dist-test-slave-s11t krb5kdc[7665](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292572, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.51.190@KRBTEST.COM for HTTP/127.21.51.148@KRBTEST.COM
I20260809 16:22:52.871275 10016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:22:52.864443 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.51.129:35083 (local address 127.21.51.131:42679)
0809 16:22:52.864592 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0809 16:22:52.864594 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:22:52.865685 (+ 1091us) server_negotiation.cc:407] Connection header received
0809 16:22:52.865830 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:22:52.865832 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:22:52.865847 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:22:52.865916 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:22:52.866094 (+ 178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:22:52.866875 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:22:52.869645 (+ 2770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:22:52.870257 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:22:52.870310 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:22:52.870319 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0809 16:22:52.871150 (+ 831us) server_negotiation.cc:299] Negotiation successful
0809 16:22:52.871200 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":113}
I20260809 16:22:52.871691 10044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:22:52.864021 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.131:42679 (local address 127.21.51.129:35083)
0809 16:22:52.865606 (+ 1585us) negotiation.cc:107] Waiting for socket to connect
0809 16:22:52.865619 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 16:22:52.865728 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:22:52.865929 (+ 201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:22:52.865930 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:22:52.865932 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:22:52.866015 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:22:52.866016 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:22:52.866986 (+ 970us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:22:52.866987 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:22:52.868354 (+ 1367us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:22:52.868358 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:22:52.870553 (+ 2195us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:22:52.870554 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:22:52.870631 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:22:52.870635 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 16:22:52.871547 (+ 912us) client_negotiation.cc:240] Negotiation successful
0809 16:22:52.871593 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1562,"thread_start_us":118,"threads_started":1}
I20260809 16:22:52.872790 9898 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67edd40372404e438b0d982421c2f163" candidate_uuid: "22ecd31dbeeb44ad8fb4a150f0af7665" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef84aefd53974155b0d5ffa6d691b55f" is_pre_election: true
I20260809 16:22:52.872956 9898 raft_consensus.cc:2454] T 67edd40372404e438b0d982421c2f163 P ef84aefd53974155b0d5ffa6d691b55f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 22ecd31dbeeb44ad8fb4a150f0af7665 in term 0.
I20260809 16:22:52.873289 9364 leader_election.cc:302] T 67edd40372404e438b0d982421c2f163 P 22ecd31dbeeb44ad8fb4a150f0af7665 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 22ecd31dbeeb44ad8fb4a150f0af7665, ef84aefd53974155b0d5ffa6d691b55f; no voters:
I20260809 16:22:52.873426 10040 raft_consensus.cc:2781] T 67edd40372404e438b0d982421c2f163 P 22ecd31dbeeb44ad8fb4a150f0af7665 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 16:22:52.873464 10040 raft_consensus.cc:482] T 67edd40372404e438b0d982421c2f163 P 22ecd31dbeeb44ad8fb4a150f0af7665 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 16:22:52.873485 10040 raft_consensus.cc:3037] T 67edd40372404e438b0d982421c2f163 P 22ecd31dbeeb44ad8fb4a150f0af7665 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:22:52.874399 10040 raft_consensus.cc:504] T 67edd40372404e438b0d982421c2f163 P 22ecd31dbeeb44ad8fb4a150f0af7665 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22ecd31dbeeb44ad8fb4a150f0af7665" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 44557 } } peers { permanent_uuid: "ef84aefd53974155b0d5ffa6d691b55f" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 42679 } } peers { permanent_uuid: "06717649045b4dc685420de2d8faec74" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 43121 } }
I20260809 16:22:52.874522 10040 leader_election.cc:288] T 67edd40372404e438b0d982421c2f163 P 22ecd31dbeeb44ad8fb4a150f0af7665 [CANDIDATE]: Term 1 election: Requested vote from peers ef84aefd53974155b0d5ffa6d691b55f (127.21.51.131:42679), 06717649045b4dc685420de2d8faec74 (127.21.51.130:43121)
I20260809 16:22:52.875094 9898 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67edd40372404e438b0d982421c2f163" candidate_uuid: "22ecd31dbeeb44ad8fb4a150f0af7665" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef84aefd53974155b0d5ffa6d691b55f"
I20260809 16:22:52.875170 9898 raft_consensus.cc:3037] T 67edd40372404e438b0d982421c2f163 P ef84aefd53974155b0d5ffa6d691b55f [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:22:52.876034 9898 raft_consensus.cc:2454] T 67edd40372404e438b0d982421c2f163 P ef84aefd53974155b0d5ffa6d691b55f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 22ecd31dbeeb44ad8fb4a150f0af7665 in term 1.
I20260809 16:22:52.876658 9364 leader_election.cc:302] T 67edd40372404e438b0d982421c2f163 P 22ecd31dbeeb44ad8fb4a150f0af7665 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 22ecd31dbeeb44ad8fb4a150f0af7665, ef84aefd53974155b0d5ffa6d691b55f; no voters:
I20260809 16:22:52.876857 10040 raft_consensus.cc:2781] T 67edd40372404e438b0d982421c2f163 P 22ecd31dbeeb44ad8fb4a150f0af7665 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:22:52.877009 10040 raft_consensus.cc:686] T 67edd40372404e438b0d982421c2f163 P 22ecd31dbeeb44ad8fb4a150f0af7665 [term 1 LEADER]: Becoming Leader. State: Replica: 22ecd31dbeeb44ad8fb4a150f0af7665, State: Running, Role: LEADER
I20260809 16:22:52.877095 10040 consensus_queue.cc:237] T 67edd40372404e438b0d982421c2f163 P 22ecd31dbeeb44ad8fb4a150f0af7665 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22ecd31dbeeb44ad8fb4a150f0af7665" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 44557 } } peers { permanent_uuid: "ef84aefd53974155b0d5ffa6d691b55f" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 42679 } } peers { permanent_uuid: "06717649045b4dc685420de2d8faec74" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 43121 } }
I20260809 16:22:52.877957 9237 catalog_manager.cc:5495] T 67edd40372404e438b0d982421c2f163 P 22ecd31dbeeb44ad8fb4a150f0af7665 reported cstate change: term changed from 0 to 1, leader changed from <none> to 22ecd31dbeeb44ad8fb4a150f0af7665 (127.21.51.129). New cstate: current_term: 1 leader_uuid: "22ecd31dbeeb44ad8fb4a150f0af7665" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22ecd31dbeeb44ad8fb4a150f0af7665" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 44557 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ef84aefd53974155b0d5ffa6d691b55f" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 42679 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "06717649045b4dc685420de2d8faec74" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 43121 } health_report { overall_health: UNKNOWN } } }
I20260809 16:22:52.879807 10042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:22:52.864236 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.130:43121 (local address 127.21.51.129:39001)
0809 16:22:52.865844 (+ 1608us) negotiation.cc:107] Waiting for socket to connect
0809 16:22:52.865852 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0809 16:22:52.868445 (+ 2593us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:22:52.868636 (+ 191us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:22:52.868637 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:22:52.868638 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:22:52.868734 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:22:52.868736 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:22:52.870697 (+ 1961us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:22:52.870698 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:22:52.878753 (+ 8055us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:22:52.878757 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:22:52.879529 (+ 772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:22:52.879530 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:22:52.879609 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:22:52.879614 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 16:22:52.879671 (+ 57us) client_negotiation.cc:240] Negotiation successful
0809 16:22:52.879720 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1594,"thread_start_us":60,"threads_started":1}
I20260809 16:22:52.879820 10019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:22:52.864499 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.51.129:39001 (local address 127.21.51.130:43121)
0809 16:22:52.864641 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0809 16:22:52.864643 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:22:52.865898 (+ 1255us) server_negotiation.cc:407] Connection header received
0809 16:22:52.868531 (+ 2633us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:22:52.868533 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:22:52.868543 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:22:52.868573 (+ 30us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:22:52.868816 (+ 243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:22:52.869537 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:22:52.878858 (+ 9321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:22:52.879424 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:22:52.879518 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:22:52.879529 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0809 16:22:52.879693 (+ 164us) server_negotiation.cc:299] Negotiation successful
0809 16:22:52.879746 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":116}
I20260809 16:22:52.880463 9658 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67edd40372404e438b0d982421c2f163" candidate_uuid: "22ecd31dbeeb44ad8fb4a150f0af7665" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06717649045b4dc685420de2d8faec74"
I20260809 16:22:52.880468 9654 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67edd40372404e438b0d982421c2f163" candidate_uuid: "22ecd31dbeeb44ad8fb4a150f0af7665" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06717649045b4dc685420de2d8faec74" is_pre_election: true
I20260809 16:22:52.880596 9658 raft_consensus.cc:3037] T 67edd40372404e438b0d982421c2f163 P 06717649045b4dc685420de2d8faec74 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:22:52.881502 9658 raft_consensus.cc:2454] T 67edd40372404e438b0d982421c2f163 P 06717649045b4dc685420de2d8faec74 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 22ecd31dbeeb44ad8fb4a150f0af7665 in term 1.
I20260809 16:22:52.918504 10024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:22:52.914992 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52332 (local address 127.21.51.129:44557)
0809 16:22:52.915046 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0809 16:22:52.915048 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:22:52.915057 (+ 9us) server_negotiation.cc:407] Connection header received
0809 16:22:52.915109 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:22:52.915109 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:22:52.915128 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:22:52.915225 (+ 97us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 16:22:52.915369 (+ 144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:22:52.916138 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:22:52.917343 (+ 1205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:22:52.917671 (+ 328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:22:52.917737 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:22:52.917983 (+ 246us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 16:22:52.918119 (+ 136us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 16:22:52.918192 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0809 16:22:52.918401 (+ 209us) server_negotiation.cc:299] Negotiation successful
0809 16:22:52.918423 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260809 16:22:52.920943 9658 raft_consensus.cc:1261] T 67edd40372404e438b0d982421c2f163 P 06717649045b4dc685420de2d8faec74 [term 1 FOLLOWER]: Refusing update from remote peer 22ecd31dbeeb44ad8fb4a150f0af7665: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 16:22:52.921304 9898 raft_consensus.cc:1261] T 67edd40372404e438b0d982421c2f163 P ef84aefd53974155b0d5ffa6d691b55f [term 1 FOLLOWER]: Refusing update from remote peer 22ecd31dbeeb44ad8fb4a150f0af7665: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 16:22:52.921650 10045 consensus_queue.cc:1035] T 67edd40372404e438b0d982421c2f163 P 22ecd31dbeeb44ad8fb4a150f0af7665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef84aefd53974155b0d5ffa6d691b55f" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 42679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 16:22:52.921768 10040 consensus_queue.cc:1035] T 67edd40372404e438b0d982421c2f163 P 22ecd31dbeeb44ad8fb4a150f0af7665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06717649045b4dc685420de2d8faec74" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 43121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260809 16:22:52.925287 9972 tablet.cc:2369] T 67edd40372404e438b0d982421c2f163 P ef84aefd53974155b0d5ffa6d691b55f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 16:22:52.927840 10055 mvcc.cc:205] Tried to move back new op lower bound from 7316654378681057280 to 7316654378505142272. Current Snapshot: MvccSnapshot[applied={T|T < 7316654378681057280}]
I20260809 16:22:52.930680 9237 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39744:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 16:22:52.930774 9237 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39744:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 16:22:52.931694 9237 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 860b05f8620c4cc2a5be9cec7d0e0fcb: Sending DeleteTablet for 3 replicas of tablet 67edd40372404e438b0d982421c2f163
I20260809 16:22:52.932126 9834 tablet_service.cc:1514] Processing DeleteTablet for tablet 67edd40372404e438b0d982421c2f163 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:22:52 UTC) from {username='kudu', principal='kudu/127.21.51.190@KRBTEST.COM'} at 127.0.0.1:43092
I20260809 16:22:52.932380 10066 tablet_replica.cc:330] T 67edd40372404e438b0d982421c2f163 P ef84aefd53974155b0d5ffa6d691b55f: stopping tablet replica
I20260809 16:22:52.932382 9649 tablet_service.cc:1514] Processing DeleteTablet for tablet 67edd40372404e438b0d982421c2f163 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:22:52 UTC) from {username='kudu', principal='kudu/127.21.51.190@KRBTEST.COM'} at 127.0.0.1:46664
I20260809 16:22:52.932479 10066 raft_consensus.cc:2229] T 67edd40372404e438b0d982421c2f163 P ef84aefd53974155b0d5ffa6d691b55f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 16:22:52.932586 10066 raft_consensus.cc:2258] T 67edd40372404e438b0d982421c2f163 P ef84aefd53974155b0d5ffa6d691b55f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 16:22:52.932812 9431 tablet_service.cc:1514] Processing DeleteTablet for tablet 67edd40372404e438b0d982421c2f163 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:22:52 UTC) from {username='kudu', principal='kudu/127.21.51.190@KRBTEST.COM'} at 127.0.0.1:52318
I20260809 16:22:52.933043 10069 tablet_replica.cc:330] T 67edd40372404e438b0d982421c2f163 P 22ecd31dbeeb44ad8fb4a150f0af7665: stopping tablet replica
I20260809 16:22:52.933535 21710 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrNqUCn/build/release/bin/kudu with pid 9287
I20260809 16:22:52.937222 10067 tablet_replica.cc:330] T 67edd40372404e438b0d982421c2f163 P 06717649045b4dc685420de2d8faec74: stopping tablet replica
I20260809 16:22:52.937392 10067 raft_consensus.cc:2229] T 67edd40372404e438b0d982421c2f163 P 06717649045b4dc685420de2d8faec74 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 16:22:52.937544 10067 pending_rounds.cc:62] T 67edd40372404e438b0d982421c2f163 P 06717649045b4dc685420de2d8faec74: Trying to abort 2 pending ops.
I20260809 16:22:52.937623 10067 pending_rounds.cc:68] T 67edd40372404e438b0d982421c2f163 P 06717649045b4dc685420de2d8faec74: Aborting op as it isn't in flight: id { term: 1 index: 2 } timestamp: 7316654378681057280 op_type: WRITE_OP write_request { tablet_id: "67edd40372404e438b0d982421c2f163" schema { columns { name: "key" type: INT32 is_key: true is_nullable: false immutable: false } columns { name: "val" type: INT32 is_key: false is_nullable: true immutable: false } } row_operations { rows: "<redacted>""\001\003\00090\000\0001\324\000\000" indirect_data: "<redacted>""" } external_consistency_mode: CLIENT_PROPAGATED authz_token { token_data: "\010\210\325\342\323\006\"8\n\ntest-admin\022*\n 0df183dee8a14068a91057148733ef1f\020\001\030\001 \001(\001" signature: "<redacted>""\226\022\347;\322\215\003\336U^P\330g\342\367\365\244~7\032\276J\256\026\247\037\325\237\002\001-\331$f\262\234\362\2569<\230X\223\200Vx\345\367\333Pl\036?I\231\363\244\236k?u:\361\206" signing_key_seq_num: 0 } } request_id { client_id: "64ec2afac71840b8ba44eebee57b70f9" seq_no: 0 first_incomplete_seq_no: 0 attempt_no: 0 }
I20260809 16:22:52.938011 10067 pending_rounds.cc:68] T 67edd40372404e438b0d982421c2f163 P 06717649045b4dc685420de2d8faec74: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7316654378505142272 op_type: NO_OP noop_request { }
I20260809 16:22:52.938791 10067 raft_consensus.cc:2866] T 67edd40372404e438b0d982421c2f163 P 06717649045b4dc685420de2d8faec74 [term 1 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260809 16:22:52.938994 10067 raft_consensus.cc:2258] T 67edd40372404e438b0d982421c2f163 P 06717649045b4dc685420de2d8faec74 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 16:22:52.941207 10066 ts_tablet_manager.cc:1905] T 67edd40372404e438b0d982421c2f163 P ef84aefd53974155b0d5ffa6d691b55f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 16:22:52.942493 10067 ts_tablet_manager.cc:1905] T 67edd40372404e438b0d982421c2f163 P 06717649045b4dc685420de2d8faec74: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 16:22:52.943254 10066 ts_tablet_manager.cc:1918] T 67edd40372404e438b0d982421c2f163 P ef84aefd53974155b0d5ffa6d691b55f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 16:22:52.943346 10066 log.cc:1196] T 67edd40372404e438b0d982421c2f163 P ef84aefd53974155b0d5ffa6d691b55f: Deleting WAL directory at /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292417722232-21710-0/minicluster-data/ts-2/wal/wals/67edd40372404e438b0d982421c2f163
I20260809 16:22:52.943674 10066 ts_tablet_manager.cc:1939] T 67edd40372404e438b0d982421c2f163 P ef84aefd53974155b0d5ffa6d691b55f: Deleting consensus metadata
I20260809 16:22:52.944151 9217 catalog_manager.cc:4841] TS ef84aefd53974155b0d5ffa6d691b55f (127.21.51.131:42679): tablet 67edd40372404e438b0d982421c2f163 (table test-table [id=0df183dee8a14068a91057148733ef1f]) successfully deleted
I20260809 16:22:52.944525 10067 ts_tablet_manager.cc:1918] T 67edd40372404e438b0d982421c2f163 P 06717649045b4dc685420de2d8faec74: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 16:22:52.944621 10067 log.cc:1196] T 67edd40372404e438b0d982421c2f163 P 06717649045b4dc685420de2d8faec74: Deleting WAL directory at /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292417722232-21710-0/minicluster-data/ts-1/wal/wals/67edd40372404e438b0d982421c2f163
I20260809 16:22:52.944896 10067 ts_tablet_manager.cc:1939] T 67edd40372404e438b0d982421c2f163 P 06717649045b4dc685420de2d8faec74: Deleting consensus metadata
I20260809 16:22:52.945379 9217 catalog_manager.cc:4841] TS 06717649045b4dc685420de2d8faec74 (127.21.51.130:43121): tablet 67edd40372404e438b0d982421c2f163 (table test-table [id=0df183dee8a14068a91057148733ef1f]) successfully deleted
W20260809 16:22:52.948030 9572 connection.cc:535] server connection from 127.21.51.129:39001 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260809 16:22:52.948227 9214 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.21.51.129:44557 (error 108)
I20260809 16:22:52.948444 21710 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrNqUCn/build/release/bin/kudu with pid 9547
I20260809 16:22:52.954679 10017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:22:52.954533 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.129:44557 (local address 127.0.0.1:52334)
0809 16:22:52.954589 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0809 16:22:52.954646 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.51.129:44557: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260809 16:22:52.954829 9214 catalog_manager.cc:4584] TS 22ecd31dbeeb44ad8fb4a150f0af7665 (127.21.51.129:44557): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 67edd40372404e438b0d982421c2f163: Network error: Client connection negotiation failed: client connection to 127.21.51.129:44557: connect: Connection refused (error 111)
I20260809 16:22:52.959385 21710 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrNqUCn/build/release/bin/kudu with pid 9760
I20260809 16:22:52.976186 21710 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrNqUCn/build/release/bin/kudu with pid 9188
I20260809 16:22:52.991052 21710 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260809 16:22:53.335950 21710 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260809 16:22:53.336050 21710 mini_ranger.cc:66] Stopping Ranger...
I20260809 16:22:53.700836 21710 mini_ranger.cc:68] Stopped Ranger
2026-08-09 16:22:53.701 UTC [7724] LOG: received smart shutdown request
2026-08-09 16:22:53.705 UTC [7724] LOG: background worker "logical replication launcher" (PID 7733) exited with exit code 1
2026-08-09 16:22:53.705 UTC [7728] LOG: shutting down
2026-08-09 16:22:53.853 UTC [7724] LOG: database system is shut down
2026-08-09T16:22:53Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (49075 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786292417722232-21710-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 16:22:54 dist-test-slave-s11t krb5kdc[10084](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 16:22:54 dist-test-slave-s11t krb5kdc[10084](info): set up 2 sockets
Aug 09 16:22:54 dist-test-slave-s11t krb5kdc[10084](info): commencing operation
krb5kdc: starting...
W20260809 16:22:55.212852 21710 mini_kdc.cc:121] Time spent starting KDC: real 1.220s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786292417722232-21710-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786292417722232-21710-0/krb5kdc/test-user.keytab.
Aug 09 16:22:55 dist-test-slave-s11t krb5kdc[10084](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292575, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T16:22:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T16:22:55Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.51.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.51.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.51.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.190.keytab.
Entry for principal kudu/127.21.51.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.190.keytab.
I20260809 16:22:55.331794 21710 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrNqUCn/build/release/bin/kudu
/tmp/dist-test-taskrNqUCn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786292417722232-21710-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786292417722232-21710-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786292417722232-21710-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786292417722232-21710-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.51.190:37071
--webserver_interface=127.21.51.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.190.keytab
--principal=kudu/127.21.51.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.51.148:43865
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.51.190:37071,127.21.51.189:42617,127.21.51.188:39369
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786292417722232-21710-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786292417722232-21710-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786292417722232-21710-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:22:55.414543 10109 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:22:55.414784 10109 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:22:55.414834 10109 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:22:55.415961 10109 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 16:22:55.416031 10109 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:22:55.416055 10109 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:22:55.416077 10109 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 16:22:55.416106 10109 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 16:22:55.417598 10109 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.51.148:43865
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786292417722232-21710-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786292417722232-21710-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.51.190:37071,127.21.51.189:42617,127.21.51.188:39369
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.190.keytab
--principal=kudu/127.21.51.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.51.190:37071
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786292417722232-21710-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.51.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10109
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786292417722232-21710-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:22:55.417881 10109 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:22:55.418102 10109 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:22:55.423161 10114 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:22:55.423194 10116 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:22:55.423194 10119 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:22:55.423352 10109 server_base.cc:1034] running on GCE node
I20260809 16:22:55.423707 10109 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:22:55.423964 10109 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:22:55.425124 10109 hybrid_clock.cc:648] HybridClock initialized: now 1786292575425106 us; error 48 us; skew 500 ppm
Aug 09 16:22:55 dist-test-slave-s11t krb5kdc[10084](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292575, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.51.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:22:55.430371 10109 init.cc:377] Logged in from keytab as kudu/127.21.51.190@KRBTEST.COM (short username kudu)
I20260809 16:22:55.430886 10109 webserver.cc:460] Webserver started at http://127.21.51.190:39823/ using document root <none> and password file <none>
I20260809 16:22:55.431113 10109 fs_manager.cc:359] Metadata directory not provided
I20260809 16:22:55.431181 10109 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:22:55.431341 10109 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:22:55.432384 10109 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786292417722232-21710-0/minicluster-data/master-0/data/instance:
uuid: "38811d92d6db4a369472a4df011205fe"
format_stamp: "Formatted at 2026-08-09 16:22:55 on dist-test-slave-s11t"
server_key: "98b3abaab35e44e2b5c2b23038d56098"
server_key_iv: "189352530979ae03c4f7817162d7ab2e"
server_key_version: "encryptionkey@0"
I20260809 16:22:55.432829 10109 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786292417722232-21710-0/minicluster-data/master-0/wal/instance:
uuid: "38811d92d6db4a369472a4df011205fe"
format_stamp: "Formatted at 2026-08-09 16:22:55 on dist-test-slave-s11t"
server_key: "98b3abaab35e44e2b5c2b23038d56098"
server_key_iv: "189352530979ae03c4f7817162d7ab2e"
server_key_version: "encryptionkey@0"
I20260809 16:22:55.434173 10109 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:22:55.434955 10126 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:22:55.435137 10109 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:22:55.435242 10109 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786292417722232-21710-0/minicluster-data/master-0/data,/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786292417722232-21710-0/minicluster-data/master-0/wal
uuid: "38811d92d6db4a369472a4df011205fe"
format_stamp: "Formatted at 2026-08-09 16:22:55 on dist-test-slave-s11t"
server_key: "98b3abaab35e44e2b5c2b23038d56098"
server_key_iv: "189352530979ae03c4f7817162d7ab2e"
server_key_version: "encryptionkey@0"
I20260809 16:22:55.435329 10109 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786292417722232-21710-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786292417722232-21710-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786292417722232-21710-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:22:55.445744 10109 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:22:55.447538 10109 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:22:55.447702 10109 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:22:55.451999 10109 rpc_server.cc:307] RPC server started. Bound to: 127.21.51.190:37071
I20260809 16:22:55.452033 10205 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.51.190:37071 every 8 connection(s)
I20260809 16:22:55.452373 10109 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786292417722232-21710-0/minicluster-data/master-0/data/info.pb
I20260809 16:22:55.453297 10206 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:22:55.454813 10206 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.51.190" port: 37071 } has no permanent_uuid. Determining permanent_uuid...
I20260809 16:22:55.456645 21710 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrNqUCn/build/release/bin/kudu as pid 10109
I20260809 16:22:55.456804 21710 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786292417722232-21710-0/minicluster-data/master-0/wal/instance
I20260809 16:22:55.457001 21710 external_mini_cluster.cc:1428] Setting key b299818099746ec89fe8981a12ff4ab2
Aug 09 16:22:55 dist-test-slave-s11t krb5kdc[10084](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292575, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.51.190@KRBTEST.COM for kudu/127.21.51.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.21.51.189@KRBTEST.COM; defaulting to no policy
I20260809 16:22:55.465529 10208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:22:55.455496 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:37071 (local address 127.0.0.1:44450)
0809 16:22:55.455672 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0809 16:22:55.455707 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0809 16:22:55.456148 (+ 441us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:22:55.456429 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:22:55.456432 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:22:55.456733 (+ 301us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:22:55.456973 (+ 240us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:22:55.456986 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:22:55.458057 (+ 1071us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:22:55.458059 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:22:55.459080 (+ 1021us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:22:55.459083 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:22:55.459778 (+ 695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:22:55.459779 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:22:55.459839 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:22:55.460345 (+ 506us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:22:55.460353 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:22:55.462033 (+ 1680us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:22:55.463901 (+ 1868us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:22:55.463908 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:22:55.463910 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:22:55.464106 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:22:55.464374 (+ 268us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:22:55.464375 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:22:55.464375 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:22:55.464465 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:22:55.464889 (+ 424us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:22:55.464895 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:22:55.464959 (+ 64us) client_negotiation.cc:769] Sending connection context
0809 16:22:55.465070 (+ 111us) client_negotiation.cc:240] Negotiation successful
0809 16:22:55.465333 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"mutex_wait_us":124,"thread_start_us":88,"threads_started":1}
I20260809 16:22:55.465531 10210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:22:55.455531 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44450 (local address 127.21.51.190:37071)
0809 16:22:55.455731 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0809 16:22:55.455734 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 16:22:55.455776 (+ 42us) server_negotiation.cc:407] Connection header received
0809 16:22:55.456279 (+ 503us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:22:55.456281 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:22:55.456296 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:22:55.456383 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:22:55.457110 (+ 727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:22:55.457949 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:22:55.459169 (+ 1220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:22:55.459686 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:22:55.459826 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:22:55.462125 (+ 2299us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:22:55.462132 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:22:55.462133 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:22:55.462141 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:22:55.463787 (+ 1646us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:22:55.464209 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:22:55.464212 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:22:55.464213 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:22:55.464297 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:22:55.464537 (+ 240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:22:55.464538 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:22:55.464538 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:22:55.464813 (+ 275us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:22:55.464906 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0809 16:22:55.465096 (+ 190us) server_negotiation.cc:299] Negotiation successful
0809 16:22:55.465284 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":19,"server-negotiator.queue_time_us":178,"thread_start_us":71,"threads_started":1}
I20260809 16:22:55.466320 10206 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.51.189" port: 42617 } has no permanent_uuid. Determining permanent_uuid...
I20260809 16:22:55.466641 10208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:22:55.466535 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.189:42617 (local address 127.0.0.1:57698)
0809 16:22:55.466574 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0809 16:22:55.466618 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.51.189:42617: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
W20260809 16:22:55.466866 10127 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.21.51.189:42617: connect: Connection refused (error 111)
I20260809 16:22:55.467317 10208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:22:55.467221 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.189:42617 (local address 127.0.0.1:57708)
0809 16:22:55.467264 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0809 16:22:55.467294 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.51.189:42617: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260809 16:22:55.467490 10206 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.51.189:42617: Network error: Client connection negotiation failed: client connection to 127.21.51.189:42617: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.51.189@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.51.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.189.keytab.
Entry for principal kudu/127.21.51.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.189.keytab.
I20260809 16:22:55.481535 21710 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrNqUCn/build/release/bin/kudu
/tmp/dist-test-taskrNqUCn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786292417722232-21710-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786292417722232-21710-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786292417722232-21710-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786292417722232-21710-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.51.189:42617
--webserver_interface=127.21.51.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.189.keytab
--principal=kudu/127.21.51.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.51.148:43865
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.51.190:37071,127.21.51.189:42617,127.21.51.188:39369
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786292417722232-21710-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786292417722232-21710-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786292417722232-21710-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 16:22:55.516861 10206 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.51.189" port: 42617 } attempt: 1
I20260809 16:22:55.517457 10208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:22:55.517267 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.189:42617 (local address 127.0.0.1:57724)
0809 16:22:55.517350 (+ 83us) negotiation.cc:107] Waiting for socket to connect
0809 16:22:55.517407 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.51.189:42617: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260809 16:22:55.517874 10208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:22:55.517758 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.189:42617 (local address 127.0.0.1:57728)
0809 16:22:55.517810 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0809 16:22:55.517846 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.51.189:42617: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260809 16:22:55.518005 10206 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.51.189:42617: Network error: Client connection negotiation failed: client connection to 127.21.51.189:42617: connect: Connection refused (error 111)
W20260809 16:22:55.563697 10214 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:22:55.563884 10214 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:22:55.563906 10214 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:22:55.565106 10214 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 16:22:55.565147 10214 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:22:55.565162 10214 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:22:55.565197 10214 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 16:22:55.565233 10214 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 16:22:55.566708 10214 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.51.148:43865
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786292417722232-21710-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786292417722232-21710-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.21.51.190:37071,127.21.51.189:42617,127.21.51.188:39369
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.189.keytab
--principal=kudu/127.21.51.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.51.189:42617
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786292417722232-21710-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.21.51.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10214
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786292417722232-21710-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:22:55.566954 10214 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:22:55.567178 10214 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 16:22:55.572134 10214 server_base.cc:1034] running on GCE node
W20260809 16:22:55.572114 10220 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:22:55.572095 10221 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:22:55.572113 10226 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:22:55.572563 10214 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:22:55.572891 10214 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:22:55.574059 10214 hybrid_clock.cc:648] HybridClock initialized: now 1786292575574039 us; error 40 us; skew 500 ppm
Aug 09 16:22:55 dist-test-slave-s11t krb5kdc[10084](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292575, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.51.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:22:55.576540 10214 init.cc:377] Logged in from keytab as kudu/127.21.51.189@KRBTEST.COM (short username kudu)
I20260809 16:22:55.577034 10214 webserver.cc:460] Webserver started at http://127.21.51.189:43873/ using document root <none> and password file <none>
I20260809 16:22:55.577255 10214 fs_manager.cc:359] Metadata directory not provided
I20260809 16:22:55.577322 10214 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:22:55.577467 10214 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:22:55.578454 10214 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786292417722232-21710-0/minicluster-data/master-1/data/instance:
uuid: "fdda3347c15444659357d8e9bfce7afa"
format_stamp: "Formatted at 2026-08-09 16:22:55 on dist-test-slave-s11t"
server_key: "163edb39a2dd51ad0f8b6230f2eb81fc"
server_key_iv: "b24a7640089ec5f4f4887f41c656f03e"
server_key_version: "encryptionkey@0"
I20260809 16:22:55.578824 10214 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786292417722232-21710-0/minicluster-data/master-1/wal/instance:
uuid: "fdda3347c15444659357d8e9bfce7afa"
format_stamp: "Formatted at 2026-08-09 16:22:55 on dist-test-slave-s11t"
server_key: "163edb39a2dd51ad0f8b6230f2eb81fc"
server_key_iv: "b24a7640089ec5f4f4887f41c656f03e"
server_key_version: "encryptionkey@0"
I20260809 16:22:55.580181 10214 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 16:22:55.580905 10240 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:22:55.581077 10214 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 16:22:55.581179 10214 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786292417722232-21710-0/minicluster-data/master-1/data,/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786292417722232-21710-0/minicluster-data/master-1/wal
uuid: "fdda3347c15444659357d8e9bfce7afa"
format_stamp: "Formatted at 2026-08-09 16:22:55 on dist-test-slave-s11t"
server_key: "163edb39a2dd51ad0f8b6230f2eb81fc"
server_key_iv: "b24a7640089ec5f4f4887f41c656f03e"
server_key_version: "encryptionkey@0"
I20260809 16:22:55.581249 10214 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786292417722232-21710-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786292417722232-21710-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786292417722232-21710-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:22:55.586174 10206 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.51.189" port: 42617 } attempt: 2
I20260809 16:22:55.586567 10208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:22:55.586435 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.189:42617 (local address 127.0.0.1:57744)
0809 16:22:55.586492 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0809 16:22:55.586532 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.51.189:42617: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260809 16:22:55.586923 10208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:22:55.586826 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.189:42617 (local address 127.0.0.1:57748)
0809 16:22:55.586873 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0809 16:22:55.586900 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.51.189:42617: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260809 16:22:55.587031 10206 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.51.189:42617: Network error: Client connection negotiation failed: client connection to 127.21.51.189:42617: connect: Connection refused (error 111)
I20260809 16:22:55.592651 10214 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:22:55.594712 10214 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:22:55.594887 10214 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:22:55.599062 10214 rpc_server.cc:307] RPC server started. Bound to: 127.21.51.189:42617
I20260809 16:22:55.599148 10323 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.51.189:42617 every 8 connection(s)
I20260809 16:22:55.599457 10214 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786292417722232-21710-0/minicluster-data/master-1/data/info.pb
I20260809 16:22:55.600309 10324 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:22:55.601879 10324 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.51.190" port: 37071 } has no permanent_uuid. Determining permanent_uuid...
I20260809 16:22:55.606369 21710 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrNqUCn/build/release/bin/kudu as pid 10214
I20260809 16:22:55.606470 21710 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786292417722232-21710-0/minicluster-data/master-1/wal/instance
I20260809 16:22:55.606706 21710 external_mini_cluster.cc:1428] Setting key 3c14f11388f77b8725a1481ad8c1abd6
Aug 09 16:22:55 dist-test-slave-s11t krb5kdc[10084](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292575, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.51.189@KRBTEST.COM for kudu/127.21.51.190@KRBTEST.COM
I20260809 16:22:55.611091 10326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:22:55.602505 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:37071 (local address 127.0.0.1:44464)
0809 16:22:55.602704 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0809 16:22:55.602740 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0809 16:22:55.603177 (+ 437us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:22:55.603397 (+ 220us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:22:55.603400 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:22:55.603631 (+ 231us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:22:55.603817 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:22:55.603829 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:22:55.604497 (+ 668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:22:55.604498 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:22:55.605360 (+ 862us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:22:55.605362 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:22:55.605709 (+ 347us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:22:55.605710 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:22:55.605810 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:22:55.606264 (+ 454us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:22:55.606272 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:22:55.607753 (+ 1481us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:22:55.609675 (+ 1922us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:22:55.609682 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:22:55.609684 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:22:55.609912 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:22:55.610119 (+ 207us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:22:55.610120 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:22:55.610120 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:22:55.610216 (+ 96us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:22:55.610541 (+ 325us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:22:55.610547 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:22:55.610709 (+ 162us) client_negotiation.cc:769] Sending connection context
0809 16:22:55.610817 (+ 108us) client_negotiation.cc:240] Negotiation successful
0809 16:22:55.610938 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":103,"threads_started":1}
I20260809 16:22:55.611096 10210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:22:55.602553 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44464 (local address 127.21.51.190:37071)
0809 16:22:55.602630 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0809 16:22:55.602632 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:22:55.602821 (+ 189us) server_negotiation.cc:407] Connection header received
0809 16:22:55.603256 (+ 435us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:22:55.603258 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:22:55.603268 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:22:55.603339 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:22:55.603912 (+ 573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:22:55.604411 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:22:55.605433 (+ 1022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:22:55.605633 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:22:55.605689 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:22:55.607872 (+ 2183us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:22:55.607874 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:22:55.607874 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:22:55.607882 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:22:55.609577 (+ 1695us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:22:55.609983 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:22:55.609984 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:22:55.609985 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:22:55.610050 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:22:55.610279 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:22:55.610280 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:22:55.610281 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:22:55.610440 (+ 159us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:22:55.610512 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0809 16:22:55.610862 (+ 350us) server_negotiation.cc:299] Negotiation successful
0809 16:22:55.610958 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260809 16:22:55.612213 10324 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.51.189" port: 42617 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.21.51.188@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.51.188@KRBTEST.COM" created.
Aug 09 16:22:55 dist-test-slave-s11t krb5kdc[10084](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292575, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.51.189@KRBTEST.COM for kudu/127.21.51.189@KRBTEST.COM
I20260809 16:22:55.622864 10326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:22:55.612464 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.189:42617 (local address 127.0.0.1:57760)
0809 16:22:55.612512 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0809 16:22:55.612523 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 16:22:55.612564 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:22:55.612878 (+ 314us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:22:55.612879 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:22:55.613041 (+ 162us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:22:55.613102 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:22:55.613104 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:22:55.613960 (+ 856us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:22:55.613961 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:22:55.614791 (+ 830us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:22:55.614794 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:22:55.615294 (+ 500us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:22:55.615296 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:22:55.615391 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:22:55.615775 (+ 384us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:22:55.615780 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:22:55.619526 (+ 3746us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:22:55.621720 (+ 2194us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:22:55.621721 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:22:55.621721 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:22:55.621910 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:22:55.622134 (+ 224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:22:55.622135 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:22:55.622136 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:22:55.622191 (+ 55us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:22:55.622483 (+ 292us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:22:55.622484 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:22:55.622523 (+ 39us) client_negotiation.cc:769] Sending connection context
0809 16:22:55.622603 (+ 80us) client_negotiation.cc:240] Negotiation successful
0809 16:22:55.622741 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31}
I20260809 16:22:55.622970 10329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:22:55.612553 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57760 (local address 127.21.51.189:42617)
0809 16:22:55.612711 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0809 16:22:55.612716 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 16:22:55.612736 (+ 20us) server_negotiation.cc:407] Connection header received
0809 16:22:55.612757 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:22:55.612759 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:22:55.612773 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:22:55.612851 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:22:55.613205 (+ 354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:22:55.613869 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:22:55.614888 (+ 1019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:22:55.615205 (+ 317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:22:55.615269 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:22:55.619635 (+ 4366us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:22:55.619642 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:22:55.619642 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:22:55.619648 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:22:55.621637 (+ 1989us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:22:55.622003 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:22:55.622006 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:22:55.622006 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:22:55.622065 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:22:55.622254 (+ 189us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:22:55.622255 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:22:55.622255 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:22:55.622404 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:22:55.622494 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0809 16:22:55.622637 (+ 143us) server_negotiation.cc:299] Negotiation successful
0809 16:22:55.622821 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":93,"server-negotiator.queue_time_us":118,"thread_start_us":91,"threads_started":1}
I20260809 16:22:55.623713 10324 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.51.188" port: 39369 } has no permanent_uuid. Determining permanent_uuid...
I20260809 16:22:55.624063 10326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:22:55.623940 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.188:39369 (local address 127.0.0.1:38522)
0809 16:22:55.623988 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0809 16:22:55.624032 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.51.188:39369: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260809 16:22:55.624229 10243 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.21.51.188:39369: connect: Connection refused (error 111)
I20260809 16:22:55.624732 10326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:22:55.624598 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.188:39369 (local address 127.0.0.1:38534)
0809 16:22:55.624649 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0809 16:22:55.624673 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.51.188:39369: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260809 16:22:55.624848 10324 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.51.188:39369: Network error: Client connection negotiation failed: client connection to 127.21.51.188:39369: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.51.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.188.keytab.
Entry for principal kudu/127.21.51.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.188.keytab.
I20260809 16:22:55.633361 21710 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrNqUCn/build/release/bin/kudu
/tmp/dist-test-taskrNqUCn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786292417722232-21710-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786292417722232-21710-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786292417722232-21710-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786292417722232-21710-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.51.188:39369
--webserver_interface=127.21.51.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.188.keytab
--principal=kudu/127.21.51.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.51.148:43865
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.51.190:37071,127.21.51.189:42617,127.21.51.188:39369
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786292417722232-21710-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786292417722232-21710-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786292417722232-21710-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 16:22:55.674093 10324 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.51.188" port: 39369 } attempt: 1
I20260809 16:22:55.674660 10326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:22:55.674515 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.188:39369 (local address 127.0.0.1:38550)
0809 16:22:55.674587 (+ 72us) negotiation.cc:107] Waiting for socket to connect
0809 16:22:55.674631 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.51.188:39369: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260809 16:22:55.675066 10326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:22:55.674984 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.188:39369 (local address 127.0.0.1:38556)
0809 16:22:55.675025 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0809 16:22:55.675050 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.51.188:39369: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260809 16:22:55.675174 10324 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.51.188:39369: Network error: Client connection negotiation failed: client connection to 127.21.51.188:39369: connect: Connection refused (error 111)
I20260809 16:22:55.678200 10206 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.51.189" port: 42617 } attempt: 3
Aug 09 16:22:55 dist-test-slave-s11t krb5kdc[10084](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292575, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.51.190@KRBTEST.COM for kudu/127.21.51.189@KRBTEST.COM
I20260809 16:22:55.685467 10329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:22:55.678592 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57764 (local address 127.21.51.189:42617)
0809 16:22:55.678639 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0809 16:22:55.678641 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:22:55.678652 (+ 11us) server_negotiation.cc:407] Connection header received
0809 16:22:55.678748 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:22:55.678749 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:22:55.678759 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:22:55.678835 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:22:55.679173 (+ 338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:22:55.679696 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:22:55.680423 (+ 727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:22:55.680635 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:22:55.680743 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:22:55.682617 (+ 1874us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:22:55.682618 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:22:55.682619 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:22:55.682627 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:22:55.684181 (+ 1554us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:22:55.684602 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:22:55.684603 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:22:55.684603 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:22:55.684666 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:22:55.684937 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:22:55.684938 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:22:55.684938 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:22:55.685056 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:22:55.685131 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0809 16:22:55.685302 (+ 171us) server_negotiation.cc:299] Negotiation successful
0809 16:22:55.685382 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260809 16:22:55.685472 10208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:22:55.678533 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.189:42617 (local address 127.0.0.1:57764)
0809 16:22:55.678599 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0809 16:22:55.678611 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 16:22:55.678668 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:22:55.678842 (+ 174us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:22:55.678843 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:22:55.679013 (+ 170us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:22:55.679087 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:22:55.679089 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:22:55.679783 (+ 694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:22:55.679784 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:22:55.680301 (+ 517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:22:55.680303 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:22:55.680741 (+ 438us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:22:55.680742 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:22:55.680831 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:22:55.681297 (+ 466us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:22:55.681310 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:22:55.682526 (+ 1216us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:22:55.684269 (+ 1743us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:22:55.684270 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:22:55.684271 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:22:55.684528 (+ 257us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:22:55.684772 (+ 244us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:22:55.684773 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:22:55.684773 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:22:55.684882 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:22:55.685133 (+ 251us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:22:55.685134 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:22:55.685182 (+ 48us) client_negotiation.cc:769] Sending connection context
0809 16:22:55.685264 (+ 82us) client_negotiation.cc:240] Negotiation successful
0809 16:22:55.685372 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":46}
I20260809 16:22:55.686081 10206 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.51.188" port: 39369 } has no permanent_uuid. Determining permanent_uuid...
I20260809 16:22:55.686429 10208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:22:55.686326 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.188:39369 (local address 127.0.0.1:38572)
0809 16:22:55.686371 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0809 16:22:55.686405 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.51.188:39369: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
I20260809 16:22:55.686767 10208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:22:55.686706 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.188:39369 (local address 127.0.0.1:38576)
0809 16:22:55.686729 (+ 23us) negotiation.cc:107] Waiting for socket to connect
0809 16:22:55.686751 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.51.188:39369: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":14}
W20260809 16:22:55.686875 10206 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.51.188:39369: Network error: Client connection negotiation failed: client connection to 127.21.51.188:39369: connect: Connection refused (error 111)
W20260809 16:22:55.716749 10335 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:22:55.716941 10335 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:22:55.716964 10335 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260809 16:22:55.718003 10206 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.51.188" port: 39369 } attempt: 1
W20260809 16:22:55.718164 10335 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 16:22:55.718216 10335 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:22:55.718242 10335 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:22:55.718292 10335 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 16:22:55.718312 10335 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 16:22:55.718452 10208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:22:55.718333 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.188:39369 (local address 127.0.0.1:38580)
0809 16:22:55.718388 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0809 16:22:55.718426 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.51.188:39369: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260809 16:22:55.718895 10208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:22:55.718800 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.188:39369 (local address 127.0.0.1:38596)
0809 16:22:55.718847 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0809 16:22:55.718878 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.51.188:39369: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260809 16:22:55.719012 10206 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.51.188:39369: Network error: Client connection negotiation failed: client connection to 127.21.51.188:39369: connect: Connection refused (error 111)
I20260809 16:22:55.720600 10335 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.51.148:43865
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786292417722232-21710-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786292417722232-21710-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.21.51.190:37071,127.21.51.189:42617,127.21.51.188:39369
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.188.keytab
--principal=kudu/127.21.51.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.51.188:39369
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786292417722232-21710-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.21.51.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10335
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786292417722232-21710-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:22:55.720927 10335 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:22:55.721170 10335 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:22:55.726536 10346 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:22:55.726536 10341 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:22:55.726557 10342 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:22:55.726851 10335 server_base.cc:1034] running on GCE node
I20260809 16:22:55.727113 10335 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:22:55.727437 10335 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:22:55.728608 10335 hybrid_clock.cc:648] HybridClock initialized: now 1786292575728599 us; error 38 us; skew 500 ppm
Aug 09 16:22:55 dist-test-slave-s11t krb5kdc[10084](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292575, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.51.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:22:55.731123 10335 init.cc:377] Logged in from keytab as kudu/127.21.51.188@KRBTEST.COM (short username kudu)
I20260809 16:22:55.731559 10335 webserver.cc:460] Webserver started at http://127.21.51.188:35095/ using document root <none> and password file <none>
I20260809 16:22:55.731781 10335 fs_manager.cc:359] Metadata directory not provided
I20260809 16:22:55.731851 10335 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:22:55.732012 10335 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:22:55.733126 10335 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786292417722232-21710-0/minicluster-data/master-2/data/instance:
uuid: "6afec1683ee64fd1952734669cd8de8a"
format_stamp: "Formatted at 2026-08-09 16:22:55 on dist-test-slave-s11t"
server_key: "999917fbbd79b5e73afcd78bd7c34501"
server_key_iv: "0bc39f65043df4eeb4f9d7bec2c7add9"
server_key_version: "encryptionkey@0"
I20260809 16:22:55.733490 10335 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786292417722232-21710-0/minicluster-data/master-2/wal/instance:
uuid: "6afec1683ee64fd1952734669cd8de8a"
format_stamp: "Formatted at 2026-08-09 16:22:55 on dist-test-slave-s11t"
server_key: "999917fbbd79b5e73afcd78bd7c34501"
server_key_iv: "0bc39f65043df4eeb4f9d7bec2c7add9"
server_key_version: "encryptionkey@0"
I20260809 16:22:55.734854 10335 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 16:22:55.735607 10355 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:22:55.735786 10335 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 16:22:55.735908 10335 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786292417722232-21710-0/minicluster-data/master-2/data,/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786292417722232-21710-0/minicluster-data/master-2/wal
uuid: "6afec1683ee64fd1952734669cd8de8a"
format_stamp: "Formatted at 2026-08-09 16:22:55 on dist-test-slave-s11t"
server_key: "999917fbbd79b5e73afcd78bd7c34501"
server_key_iv: "0bc39f65043df4eeb4f9d7bec2c7add9"
server_key_version: "encryptionkey@0"
I20260809 16:22:55.735984 10335 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786292417722232-21710-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786292417722232-21710-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786292417722232-21710-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:22:55.743309 10324 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.51.188" port: 39369 } attempt: 2
I20260809 16:22:55.743685 10326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:22:55.743559 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.188:39369 (local address 127.0.0.1:38610)
0809 16:22:55.743617 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0809 16:22:55.743652 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.51.188:39369: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260809 16:22:55.744016 10326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:22:55.743924 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.188:39369 (local address 127.0.0.1:38626)
0809 16:22:55.743967 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0809 16:22:55.743992 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.51.188:39369: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
W20260809 16:22:55.744107 10324 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.51.188:39369: Network error: Client connection negotiation failed: client connection to 127.21.51.188:39369: connect: Connection refused (error 111)
I20260809 16:22:55.746222 10335 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:22:55.748445 10335 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:22:55.748607 10335 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:22:55.752954 10335 rpc_server.cc:307] RPC server started. Bound to: 127.21.51.188:39369
I20260809 16:22:55.753006 10437 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.51.188:39369 every 8 connection(s)
I20260809 16:22:55.753433 10335 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786292417722232-21710-0/minicluster-data/master-2/data/info.pb
I20260809 16:22:55.754281 10438 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:22:55.755707 10438 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.51.190" port: 37071 } has no permanent_uuid. Determining permanent_uuid...
I20260809 16:22:55.758287 21710 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrNqUCn/build/release/bin/kudu as pid 10335
I20260809 16:22:55.758381 21710 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786292417722232-21710-0/minicluster-data/master-2/wal/instance
I20260809 16:22:55.758584 21710 external_mini_cluster.cc:1428] Setting key b3b33dd197539fcd10d6fda1fde96f2b
Aug 09 16:22:55 dist-test-slave-s11t krb5kdc[10084](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292575, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.51.188@KRBTEST.COM for kudu/127.21.51.190@KRBTEST.COM
Aug 09 16:22:55 dist-test-slave-s11t krb5kdc[10084](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292575, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.51.188@KRBTEST.COM
I20260809 16:22:55.765043 10210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:22:55.756344 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44470 (local address 127.21.51.190:37071)
0809 16:22:55.756394 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0809 16:22:55.756396 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:22:55.756609 (+ 213us) server_negotiation.cc:407] Connection header received
0809 16:22:55.757125 (+ 516us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:22:55.757126 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:22:55.757136 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:22:55.757202 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:22:55.757809 (+ 607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:22:55.758372 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:22:55.759540 (+ 1168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:22:55.759758 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:22:55.759838 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:22:55.762148 (+ 2310us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:22:55.762149 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:22:55.762150 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:22:55.762158 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:22:55.763500 (+ 1342us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:22:55.763991 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:22:55.763992 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:22:55.763992 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:22:55.764055 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:22:55.764301 (+ 246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:22:55.764302 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:22:55.764303 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:22:55.764448 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:22:55.764513 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0809 16:22:55.764863 (+ 350us) server_negotiation.cc:299] Negotiation successful
0809 16:22:55.764957 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260809 16:22:55.765208 10441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:22:55.756323 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:37071 (local address 127.0.0.1:44470)
0809 16:22:55.756491 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0809 16:22:55.756525 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0809 16:22:55.757014 (+ 489us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:22:55.757274 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:22:55.757277 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:22:55.757520 (+ 243us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:22:55.757704 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:22:55.757708 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:22:55.758483 (+ 775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:22:55.758484 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:22:55.759445 (+ 961us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:22:55.759448 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:22:55.759843 (+ 395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:22:55.759844 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:22:55.759985 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:22:55.760449 (+ 464us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:22:55.760457 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:22:55.762062 (+ 1605us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:22:55.763586 (+ 1524us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:22:55.763594 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:22:55.763596 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:22:55.763914 (+ 318us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:22:55.764128 (+ 214us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:22:55.764129 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:22:55.764129 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:22:55.764232 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:22:55.764543 (+ 311us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:22:55.764550 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:22:55.764672 (+ 122us) client_negotiation.cc:769] Sending connection context
0809 16:22:55.764829 (+ 157us) client_negotiation.cc:240] Negotiation successful
0809 16:22:55.764982 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":79,"threads_started":1}
I20260809 16:22:55.765969 10438 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.51.189" port: 42617 } has no permanent_uuid. Determining permanent_uuid...
I20260809 16:22:55.768143 10443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:22:55.758990 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38632 (local address 127.21.51.188:39369)
0809 16:22:55.759155 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0809 16:22:55.759159 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 16:22:55.759187 (+ 28us) server_negotiation.cc:407] Connection header received
0809 16:22:55.759244 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:22:55.759246 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:22:55.759266 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:22:55.759356 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:22:55.759947 (+ 591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:22:55.760882 (+ 935us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:22:55.761913 (+ 1031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:22:55.762318 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:22:55.762409 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:22:55.764966 (+ 2557us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:22:55.764974 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:22:55.764974 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:22:55.764981 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:22:55.766928 (+ 1947us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:22:55.767284 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:22:55.767287 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:22:55.767288 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:22:55.767349 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:22:55.767567 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:22:55.767568 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:22:55.767568 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:22:55.767706 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:22:55.767901 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0809 16:22:55.767958 (+ 57us) server_negotiation.cc:299] Negotiation successful
0809 16:22:55.768034 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":132,"thread_start_us":82,"threads_started":1}
Aug 09 16:22:55 dist-test-slave-s11t krb5kdc[10084](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292575, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.51.188@KRBTEST.COM for kudu/127.21.51.189@KRBTEST.COM
I20260809 16:22:55.772801 10441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:22:55.766231 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.189:42617 (local address 127.0.0.1:57780)
0809 16:22:55.766291 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0809 16:22:55.766302 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 16:22:55.766346 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:22:55.766469 (+ 123us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:22:55.766470 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:22:55.766601 (+ 131us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:22:55.766634 (+ 33us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:22:55.766636 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:22:55.767245 (+ 609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:22:55.767246 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:22:55.767717 (+ 471us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:22:55.767719 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:22:55.768221 (+ 502us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:22:55.768222 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:22:55.768294 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:22:55.768780 (+ 486us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:22:55.768785 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:22:55.770133 (+ 1348us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:22:55.771743 (+ 1610us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:22:55.771744 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:22:55.771744 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:22:55.771934 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:22:55.772153 (+ 219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:22:55.772154 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:22:55.772154 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:22:55.772213 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:22:55.772477 (+ 264us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:22:55.772478 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:22:55.772524 (+ 46us) client_negotiation.cc:769] Sending connection context
0809 16:22:55.772591 (+ 67us) client_negotiation.cc:240] Negotiation successful
0809 16:22:55.772683 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41}
I20260809 16:22:55.772809 10329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:22:55.766304 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57780 (local address 127.21.51.189:42617)
0809 16:22:55.766345 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0809 16:22:55.766346 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 16:22:55.766356 (+ 10us) server_negotiation.cc:407] Connection header received
0809 16:22:55.766410 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:22:55.766411 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:22:55.766418 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:22:55.766462 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:22:55.766705 (+ 243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:22:55.767158 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:22:55.767821 (+ 663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:22:55.768060 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:22:55.768145 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:22:55.770228 (+ 2083us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:22:55.770229 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:22:55.770230 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:22:55.770237 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:22:55.771665 (+ 1428us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:22:55.772018 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:22:55.772019 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:22:55.772020 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:22:55.772082 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:22:55.772290 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:22:55.772291 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:22:55.772291 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:22:55.772401 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:22:55.772472 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0809 16:22:55.772632 (+ 160us) server_negotiation.cc:299] Negotiation successful
0809 16:22:55.772725 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260809 16:22:55.773211 10438 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.51.188" port: 39369 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 16:22:55 dist-test-slave-s11t krb5kdc[10084](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292575, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.51.190@KRBTEST.COM
I20260809 16:22:55.776374 10210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:22:55.768924 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44476 (local address 127.21.51.190:37071)
0809 16:22:55.768962 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0809 16:22:55.768963 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 16:22:55.769011 (+ 48us) server_negotiation.cc:407] Connection header received
0809 16:22:55.769081 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:22:55.769082 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:22:55.769091 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:22:55.769146 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:22:55.769735 (+ 589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:22:55.770363 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:22:55.771360 (+ 997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:22:55.771536 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:22:55.771588 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:22:55.773865 (+ 2277us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:22:55.773866 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:22:55.773866 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:22:55.773872 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:22:55.775228 (+ 1356us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:22:55.775580 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:22:55.775581 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:22:55.775581 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:22:55.775645 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:22:55.775850 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:22:55.775851 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:22:55.775851 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:22:55.776022 (+ 171us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:22:55.776203 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0809 16:22:55.776251 (+ 48us) server_negotiation.cc:299] Negotiation successful
0809 16:22:55.776313 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
Aug 09 16:22:55 dist-test-slave-s11t krb5kdc[10084](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292575, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.51.188@KRBTEST.COM for kudu/127.21.51.188@KRBTEST.COM
I20260809 16:22:55.780457 10441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:22:55.773467 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.188:39369 (local address 127.0.0.1:38646)
0809 16:22:55.773553 (+ 86us) negotiation.cc:107] Waiting for socket to connect
0809 16:22:55.773562 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 16:22:55.773613 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:22:55.773737 (+ 124us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:22:55.773737 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:22:55.773888 (+ 151us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:22:55.773920 (+ 32us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:22:55.773921 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:22:55.774830 (+ 909us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:22:55.774831 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:22:55.775314 (+ 483us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:22:55.775315 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:22:55.775718 (+ 403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:22:55.775719 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:22:55.775813 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:22:55.776181 (+ 368us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:22:55.776186 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:22:55.777556 (+ 1370us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:22:55.779266 (+ 1710us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:22:55.779267 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:22:55.779267 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:22:55.779499 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:22:55.779698 (+ 199us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:22:55.779698 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:22:55.779699 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:22:55.779773 (+ 74us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:22:55.780240 (+ 467us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:22:55.780241 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:22:55.780278 (+ 37us) client_negotiation.cc:769] Sending connection context
0809 16:22:55.780336 (+ 58us) client_negotiation.cc:240] Negotiation successful
0809 16:22:55.780401 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":72}
I20260809 16:22:55.780601 10443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:22:55.773517 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38646 (local address 127.21.51.188:39369)
0809 16:22:55.773556 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0809 16:22:55.773558 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:22:55.773600 (+ 42us) server_negotiation.cc:407] Connection header received
0809 16:22:55.773673 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:22:55.773674 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:22:55.773682 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:22:55.773730 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:22:55.774006 (+ 276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:22:55.774744 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:22:55.775399 (+ 655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:22:55.775629 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:22:55.775701 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:22:55.777640 (+ 1939us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:22:55.777641 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:22:55.777642 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:22:55.777649 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:22:55.779119 (+ 1470us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:22:55.779576 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:22:55.779577 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:22:55.779577 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:22:55.779627 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:22:55.779848 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:22:55.779849 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:22:55.779849 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:22:55.779979 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:22:55.780052 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0809 16:22:55.780405 (+ 353us) server_negotiation.cc:299] Negotiation successful
0809 16:22:55.780486 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
Aug 09 16:22:55 dist-test-slave-s11t krb5kdc[10084](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292575, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.51.189@KRBTEST.COM
I20260809 16:22:55.782039 10438 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6afec1683ee64fd1952734669cd8de8a: Bootstrap starting.
I20260809 16:22:55.783241 10438 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6afec1683ee64fd1952734669cd8de8a: Neither blocks nor log segments found. Creating new log.
I20260809 16:22:55.783569 10438 log.cc:824] T 00000000000000000000000000000000 P 6afec1683ee64fd1952734669cd8de8a: Log is configured to *not* fsync() on all Append() calls
I20260809 16:22:55.784524 10438 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6afec1683ee64fd1952734669cd8de8a: No bootstrap required, opened a new log
I20260809 16:22:55.785112 10329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:22:55.777151 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57786 (local address 127.21.51.189:42617)
0809 16:22:55.777196 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0809 16:22:55.777199 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 16:22:55.777206 (+ 7us) server_negotiation.cc:407] Connection header received
0809 16:22:55.777226 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:22:55.777228 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:22:55.777236 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:22:55.777294 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:22:55.777851 (+ 557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:22:55.778485 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:22:55.779429 (+ 944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:22:55.779736 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:22:55.779808 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:22:55.782111 (+ 2303us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:22:55.782112 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:22:55.782112 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:22:55.782120 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:22:55.783749 (+ 1629us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:22:55.784157 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:22:55.784158 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:22:55.784158 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:22:55.784218 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:22:55.784467 (+ 249us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:22:55.784468 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:22:55.784468 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:22:55.784602 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:22:55.784858 (+ 256us) server_negotiation.cc:1035] Waiting for connection context
0809 16:22:55.784947 (+ 89us) server_negotiation.cc:299] Negotiation successful
0809 16:22:55.785027 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260809 16:22:55.786586 21710 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 16:22:55.786744 10438 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6afec1683ee64fd1952734669cd8de8a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38811d92d6db4a369472a4df011205fe" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 37071 } } peers { permanent_uuid: "fdda3347c15444659357d8e9bfce7afa" member_type: VOTER last_known_addr { host: "127.21.51.189" port: 42617 } } peers { permanent_uuid: "6afec1683ee64fd1952734669cd8de8a" member_type: VOTER last_known_addr { host: "127.21.51.188" port: 39369 } }
I20260809 16:22:55.786882 10438 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6afec1683ee64fd1952734669cd8de8a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:22:55.786919 10438 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6afec1683ee64fd1952734669cd8de8a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6afec1683ee64fd1952734669cd8de8a, State: Initialized, Role: FOLLOWER
I20260809 16:22:55.787039 10438 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6afec1683ee64fd1952734669cd8de8a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38811d92d6db4a369472a4df011205fe" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 37071 } } peers { permanent_uuid: "fdda3347c15444659357d8e9bfce7afa" member_type: VOTER last_known_addr { host: "127.21.51.189" port: 42617 } } peers { permanent_uuid: "6afec1683ee64fd1952734669cd8de8a" member_type: VOTER last_known_addr { host: "127.21.51.188" port: 39369 } }
I20260809 16:22:55.787317 10445 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6afec1683ee64fd1952734669cd8de8a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38811d92d6db4a369472a4df011205fe" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 37071 } } peers { permanent_uuid: "fdda3347c15444659357d8e9bfce7afa" member_type: VOTER last_known_addr { host: "127.21.51.189" port: 42617 } } peers { permanent_uuid: "6afec1683ee64fd1952734669cd8de8a" member_type: VOTER last_known_addr { host: "127.21.51.188" port: 39369 } } }
I20260809 16:22:55.787384 10438 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6afec1683ee64fd1952734669cd8de8a [sys.catalog]: configured and running, proceeding with master startup.
I20260809 16:22:55.787434 10445 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6afec1683ee64fd1952734669cd8de8a [sys.catalog]: This master's current role is: FOLLOWER
W20260809 16:22:55.789465 10473 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 6afec1683ee64fd1952734669cd8de8a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 16:22:55.789526 10473 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 16:22:55.789710 10335 master_runner.cc:186] Error getting master registration for 127.21.51.190:37071: OK, instance_id { permanent_uuid: "38811d92d6db4a369472a4df011205fe" instance_seqno: 1786292575450821 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 16:22:55.792771 10335 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260809 16:22:55.794145 10206 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.51.188" port: 39369 } attempt: 2
I20260809 16:22:55.796249 10210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:22:55.788376 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44488 (local address 127.21.51.190:37071)
0809 16:22:55.788413 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0809 16:22:55.788415 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:22:55.788738 (+ 323us) server_negotiation.cc:407] Connection header received
0809 16:22:55.788868 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:22:55.788869 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:22:55.788879 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:22:55.788911 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:22:55.789861 (+ 950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:22:55.790553 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:22:55.791685 (+ 1132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:22:55.791985 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:22:55.792032 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:22:55.793475 (+ 1443us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:22:55.793477 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:22:55.793477 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:22:55.793485 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:22:55.795044 (+ 1559us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:22:55.795384 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:22:55.795385 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:22:55.795385 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:22:55.795444 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:22:55.795656 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:22:55.795657 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:22:55.795657 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:22:55.795782 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:22:55.795859 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0809 16:22:55.796070 (+ 211us) server_negotiation.cc:299] Negotiation successful
0809 16:22:55.796148 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260809 16:22:55.797665 10329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:22:55.788240 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57790 (local address 127.21.51.189:42617)
0809 16:22:55.788322 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0809 16:22:55.788324 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:22:55.788439 (+ 115us) server_negotiation.cc:407] Connection header received
0809 16:22:55.788531 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:22:55.788532 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:22:55.788541 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:22:55.788572 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:22:55.789981 (+ 1409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:22:55.790761 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:22:55.791778 (+ 1017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:22:55.792060 (+ 282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:22:55.792104 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:22:55.793774 (+ 1670us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:22:55.793775 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:22:55.793775 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:22:55.793782 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:22:55.795372 (+ 1590us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:22:55.795739 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:22:55.795741 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:22:55.795741 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:22:55.795791 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:22:55.796015 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:22:55.796016 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:22:55.796017 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:22:55.796137 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:22:55.796800 (+ 663us) server_negotiation.cc:1035] Waiting for connection context
0809 16:22:55.797530 (+ 730us) server_negotiation.cc:299] Negotiation successful
0809 16:22:55.797610 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260809 16:22:55.800907 10443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:22:55.788173 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38656 (local address 127.21.51.188:39369)
0809 16:22:55.788285 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0809 16:22:55.788287 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:22:55.788296 (+ 9us) server_negotiation.cc:407] Connection header received
0809 16:22:55.788393 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:22:55.788394 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:22:55.788405 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:22:55.788436 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:22:55.790141 (+ 1705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:22:55.790833 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:22:55.796037 (+ 5204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:22:55.796324 (+ 287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:22:55.796365 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:22:55.798293 (+ 1928us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:22:55.798294 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:22:55.798294 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:22:55.798302 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:22:55.799716 (+ 1414us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:22:55.800072 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:22:55.800073 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:22:55.800073 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:22:55.800136 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:22:55.800339 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:22:55.800339 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:22:55.800340 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:22:55.800465 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:22:55.800567 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0809 16:22:55.800738 (+ 171us) server_negotiation.cc:299] Negotiation successful
0809 16:22:55.800815 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
Aug 09 16:22:55 dist-test-slave-s11t krb5kdc[10084](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292575, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.51.190@KRBTEST.COM for kudu/127.21.51.188@KRBTEST.COM
I20260809 16:22:55.806555 10478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:22:55.794723 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38672 (local address 127.21.51.188:39369)
0809 16:22:55.797020 (+ 2297us) server_negotiation.cc:206] Beginning negotiation
0809 16:22:55.797021 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 16:22:55.797029 (+ 8us) server_negotiation.cc:407] Connection header received
0809 16:22:55.797048 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:22:55.797049 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:22:55.797061 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:22:55.798536 (+ 1475us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:22:55.798570 (+ 34us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:22:55.800469 (+ 1899us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:22:55.801599 (+ 1130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:22:55.801920 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:22:55.802213 (+ 293us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:22:55.803998 (+ 1785us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:22:55.803999 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:22:55.803999 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:22:55.804004 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:22:55.805520 (+ 1516us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:22:55.805898 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:22:55.805899 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:22:55.805899 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:22:55.805950 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:22:55.806141 (+ 191us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:22:55.806142 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:22:55.806142 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:22:55.806231 (+ 89us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:22:55.806278 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0809 16:22:55.806424 (+ 146us) server_negotiation.cc:299] Negotiation successful
0809 16:22:55.806478 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2275,"thread_start_us":100,"threads_started":1}
I20260809 16:22:55.806592 10208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:22:55.794375 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.188:39369 (local address 127.0.0.1:38672)
0809 16:22:55.794572 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0809 16:22:55.794583 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 16:22:55.794623 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:22:55.797195 (+ 2572us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:22:55.797196 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:22:55.797350 (+ 154us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:22:55.797390 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:22:55.797391 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:22:55.800568 (+ 3177us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:22:55.800570 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:22:55.801378 (+ 808us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:22:55.801382 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:22:55.802017 (+ 635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:22:55.802018 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:22:55.802074 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:22:55.802407 (+ 333us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:22:55.802412 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:22:55.803901 (+ 1489us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:22:55.805599 (+ 1698us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:22:55.805600 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:22:55.805601 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:22:55.805812 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:22:55.806015 (+ 203us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:22:55.806016 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:22:55.806016 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:22:55.806081 (+ 65us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:22:55.806290 (+ 209us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:22:55.806291 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:22:55.806335 (+ 44us) client_negotiation.cc:769] Sending connection context
0809 16:22:55.806409 (+ 74us) client_negotiation.cc:240] Negotiation successful
0809 16:22:55.806499 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":178}
I20260809 16:22:55.807860 10206 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 38811d92d6db4a369472a4df011205fe: Bootstrap starting.
I20260809 16:22:55.808499 10206 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 38811d92d6db4a369472a4df011205fe: Neither blocks nor log segments found. Creating new log.
I20260809 16:22:55.808849 10206 log.cc:824] T 00000000000000000000000000000000 P 38811d92d6db4a369472a4df011205fe: Log is configured to *not* fsync() on all Append() calls
I20260809 16:22:55.809644 10206 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 38811d92d6db4a369472a4df011205fe: No bootstrap required, opened a new log
I20260809 16:22:55.811465 10206 raft_consensus.cc:348] T 00000000000000000000000000000000 P 38811d92d6db4a369472a4df011205fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38811d92d6db4a369472a4df011205fe" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 37071 } } peers { permanent_uuid: "fdda3347c15444659357d8e9bfce7afa" member_type: VOTER last_known_addr { host: "127.21.51.189" port: 42617 } } peers { permanent_uuid: "6afec1683ee64fd1952734669cd8de8a" member_type: VOTER last_known_addr { host: "127.21.51.188" port: 39369 } }
I20260809 16:22:55.811596 10206 raft_consensus.cc:374] T 00000000000000000000000000000000 P 38811d92d6db4a369472a4df011205fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:22:55.811647 10206 raft_consensus.cc:729] T 00000000000000000000000000000000 P 38811d92d6db4a369472a4df011205fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38811d92d6db4a369472a4df011205fe, State: Initialized, Role: FOLLOWER
I20260809 16:22:55.811770 10206 consensus_queue.cc:260] T 00000000000000000000000000000000 P 38811d92d6db4a369472a4df011205fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38811d92d6db4a369472a4df011205fe" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 37071 } } peers { permanent_uuid: "fdda3347c15444659357d8e9bfce7afa" member_type: VOTER last_known_addr { host: "127.21.51.189" port: 42617 } } peers { permanent_uuid: "6afec1683ee64fd1952734669cd8de8a" member_type: VOTER last_known_addr { host: "127.21.51.188" port: 39369 } }
I20260809 16:22:55.812057 10206 sys_catalog.cc:564] T 00000000000000000000000000000000 P 38811d92d6db4a369472a4df011205fe [sys.catalog]: configured and running, proceeding with master startup.
I20260809 16:22:55.812043 10480 sys_catalog.cc:455] T 00000000000000000000000000000000 P 38811d92d6db4a369472a4df011205fe [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38811d92d6db4a369472a4df011205fe" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 37071 } } peers { permanent_uuid: "fdda3347c15444659357d8e9bfce7afa" member_type: VOTER last_known_addr { host: "127.21.51.189" port: 42617 } } peers { permanent_uuid: "6afec1683ee64fd1952734669cd8de8a" member_type: VOTER last_known_addr { host: "127.21.51.188" port: 39369 } } }
I20260809 16:22:55.812161 10480 sys_catalog.cc:458] T 00000000000000000000000000000000 P 38811d92d6db4a369472a4df011205fe [sys.catalog]: This master's current role is: FOLLOWER
W20260809 16:22:55.814180 10503 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 38811d92d6db4a369472a4df011205fe: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 16:22:55.814244 10503 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 16:22:55.814489 10109 master_runner.cc:186] Error getting master registration for 127.21.51.189:42617: OK, instance_id { permanent_uuid: "fdda3347c15444659357d8e9bfce7afa" instance_seqno: 1786292575597878 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 16:22:55.816012 10109 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260809 16:22:55.835250 10324 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.51.188" port: 39369 } attempt: 3
Aug 09 16:22:55 dist-test-slave-s11t krb5kdc[10084](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292575, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.51.189@KRBTEST.COM for kudu/127.21.51.188@KRBTEST.COM
I20260809 16:22:55.842814 10478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:22:55.835592 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38682 (local address 127.21.51.188:39369)
0809 16:22:55.835638 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0809 16:22:55.835640 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:22:55.835662 (+ 22us) server_negotiation.cc:407] Connection header received
0809 16:22:55.835794 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:22:55.835795 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:22:55.835810 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:22:55.835880 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:22:55.836181 (+ 301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:22:55.836671 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:22:55.837430 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:22:55.837754 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:22:55.837842 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:22:55.840097 (+ 2255us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:22:55.840098 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:22:55.840098 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:22:55.840106 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:22:55.841621 (+ 1515us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:22:55.841990 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:22:55.841991 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:22:55.841991 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:22:55.842047 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:22:55.842266 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:22:55.842267 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:22:55.842267 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:22:55.842395 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:22:55.842468 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0809 16:22:55.842636 (+ 168us) server_negotiation.cc:299] Negotiation successful
0809 16:22:55.842730 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260809 16:22:55.842814 10326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:22:55.835560 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.188:39369 (local address 127.0.0.1:38682)
0809 16:22:55.835613 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0809 16:22:55.835625 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 16:22:55.835660 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:22:55.835886 (+ 226us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:22:55.835887 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:22:55.836049 (+ 162us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:22:55.836091 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:22:55.836093 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:22:55.836798 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:22:55.836800 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:22:55.837321 (+ 521us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:22:55.837323 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:22:55.837837 (+ 514us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:22:55.837838 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:22:55.837907 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:22:55.838362 (+ 455us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:22:55.838369 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:22:55.840008 (+ 1639us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:22:55.841735 (+ 1727us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:22:55.841736 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:22:55.841737 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:22:55.841914 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:22:55.842125 (+ 211us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:22:55.842126 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:22:55.842126 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:22:55.842194 (+ 68us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:22:55.842474 (+ 280us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:22:55.842475 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:22:55.842518 (+ 43us) client_negotiation.cc:769] Sending connection context
0809 16:22:55.842598 (+ 80us) client_negotiation.cc:240] Negotiation successful
0809 16:22:55.842716 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34}
I20260809 16:22:55.844362 10324 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fdda3347c15444659357d8e9bfce7afa: Bootstrap starting.
I20260809 16:22:55.845072 10324 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fdda3347c15444659357d8e9bfce7afa: Neither blocks nor log segments found. Creating new log.
I20260809 16:22:55.845386 10324 log.cc:824] T 00000000000000000000000000000000 P fdda3347c15444659357d8e9bfce7afa: Log is configured to *not* fsync() on all Append() calls
I20260809 16:22:55.846196 10324 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fdda3347c15444659357d8e9bfce7afa: No bootstrap required, opened a new log
I20260809 16:22:55.847676 10324 raft_consensus.cc:348] T 00000000000000000000000000000000 P fdda3347c15444659357d8e9bfce7afa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38811d92d6db4a369472a4df011205fe" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 37071 } } peers { permanent_uuid: "fdda3347c15444659357d8e9bfce7afa" member_type: VOTER last_known_addr { host: "127.21.51.189" port: 42617 } } peers { permanent_uuid: "6afec1683ee64fd1952734669cd8de8a" member_type: VOTER last_known_addr { host: "127.21.51.188" port: 39369 } }
I20260809 16:22:55.847815 10324 raft_consensus.cc:374] T 00000000000000000000000000000000 P fdda3347c15444659357d8e9bfce7afa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:22:55.847855 10324 raft_consensus.cc:729] T 00000000000000000000000000000000 P fdda3347c15444659357d8e9bfce7afa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fdda3347c15444659357d8e9bfce7afa, State: Initialized, Role: FOLLOWER
I20260809 16:22:55.847965 10324 consensus_queue.cc:260] T 00000000000000000000000000000000 P fdda3347c15444659357d8e9bfce7afa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38811d92d6db4a369472a4df011205fe" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 37071 } } peers { permanent_uuid: "fdda3347c15444659357d8e9bfce7afa" member_type: VOTER last_known_addr { host: "127.21.51.189" port: 42617 } } peers { permanent_uuid: "6afec1683ee64fd1952734669cd8de8a" member_type: VOTER last_known_addr { host: "127.21.51.188" port: 39369 } }
I20260809 16:22:55.848620 10324 sys_catalog.cc:564] T 00000000000000000000000000000000 P fdda3347c15444659357d8e9bfce7afa [sys.catalog]: configured and running, proceeding with master startup.
I20260809 16:22:55.850047 10508 sys_catalog.cc:455] T 00000000000000000000000000000000 P fdda3347c15444659357d8e9bfce7afa [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38811d92d6db4a369472a4df011205fe" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 37071 } } peers { permanent_uuid: "fdda3347c15444659357d8e9bfce7afa" member_type: VOTER last_known_addr { host: "127.21.51.189" port: 42617 } } peers { permanent_uuid: "6afec1683ee64fd1952734669cd8de8a" member_type: VOTER last_known_addr { host: "127.21.51.188" port: 39369 } } }
I20260809 16:22:55.850335 10508 sys_catalog.cc:458] T 00000000000000000000000000000000 P fdda3347c15444659357d8e9bfce7afa [sys.catalog]: This master's current role is: FOLLOWER
W20260809 16:22:55.850733 10522 catalog_manager.cc:1558] T 00000000000000000000000000000000 P fdda3347c15444659357d8e9bfce7afa: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 16:22:55.850837 10522 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 16:22:55.852846 10214 master_runner.cc:263] No leader master found from master fdda3347c15444659357d8e9bfce7afa
I20260809 16:22:55.852901 10214 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260809 16:22:55.948485 10135 tablet.cc:2369] T 00000000000000000000000000000000 P 38811d92d6db4a369472a4df011205fe: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 16:22:55.999238 10365 tablet.cc:2369] T 00000000000000000000000000000000 P 6afec1683ee64fd1952734669cd8de8a: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 16:22:56.095662 10249 tablet.cc:2369] T 00000000000000000000000000000000 P fdda3347c15444659357d8e9bfce7afa: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 16:22:56.141870 10480 raft_consensus.cc:482] T 00000000000000000000000000000000 P 38811d92d6db4a369472a4df011205fe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 16:22:56.142031 10480 raft_consensus.cc:504] T 00000000000000000000000000000000 P 38811d92d6db4a369472a4df011205fe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38811d92d6db4a369472a4df011205fe" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 37071 } } peers { permanent_uuid: "fdda3347c15444659357d8e9bfce7afa" member_type: VOTER last_known_addr { host: "127.21.51.189" port: 42617 } } peers { permanent_uuid: "6afec1683ee64fd1952734669cd8de8a" member_type: VOTER last_known_addr { host: "127.21.51.188" port: 39369 } }
I20260809 16:22:56.142315 10480 leader_election.cc:288] T 00000000000000000000000000000000 P 38811d92d6db4a369472a4df011205fe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fdda3347c15444659357d8e9bfce7afa (127.21.51.189:42617), 6afec1683ee64fd1952734669cd8de8a (127.21.51.188:39369)
I20260809 16:22:56.142683 10405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "38811d92d6db4a369472a4df011205fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6afec1683ee64fd1952734669cd8de8a" is_pre_election: true
I20260809 16:22:56.142727 10283 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "38811d92d6db4a369472a4df011205fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fdda3347c15444659357d8e9bfce7afa" is_pre_election: true
I20260809 16:22:56.142845 10405 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 6afec1683ee64fd1952734669cd8de8a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38811d92d6db4a369472a4df011205fe in term 0.
I20260809 16:22:56.142876 10283 raft_consensus.cc:2454] T 00000000000000000000000000000000 P fdda3347c15444659357d8e9bfce7afa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38811d92d6db4a369472a4df011205fe in term 0.
I20260809 16:22:56.143044 10130 leader_election.cc:302] T 00000000000000000000000000000000 P 38811d92d6db4a369472a4df011205fe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38811d92d6db4a369472a4df011205fe, 6afec1683ee64fd1952734669cd8de8a; no voters:
I20260809 16:22:56.143219 10480 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 38811d92d6db4a369472a4df011205fe [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 16:22:56.143285 10480 raft_consensus.cc:482] T 00000000000000000000000000000000 P 38811d92d6db4a369472a4df011205fe [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 16:22:56.143302 10480 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 38811d92d6db4a369472a4df011205fe [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:22:56.144094 10480 raft_consensus.cc:504] T 00000000000000000000000000000000 P 38811d92d6db4a369472a4df011205fe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38811d92d6db4a369472a4df011205fe" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 37071 } } peers { permanent_uuid: "fdda3347c15444659357d8e9bfce7afa" member_type: VOTER last_known_addr { host: "127.21.51.189" port: 42617 } } peers { permanent_uuid: "6afec1683ee64fd1952734669cd8de8a" member_type: VOTER last_known_addr { host: "127.21.51.188" port: 39369 } }
I20260809 16:22:56.144239 10480 leader_election.cc:288] T 00000000000000000000000000000000 P 38811d92d6db4a369472a4df011205fe [CANDIDATE]: Term 1 election: Requested vote from peers fdda3347c15444659357d8e9bfce7afa (127.21.51.189:42617), 6afec1683ee64fd1952734669cd8de8a (127.21.51.188:39369)
I20260809 16:22:56.144414 10405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "38811d92d6db4a369472a4df011205fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6afec1683ee64fd1952734669cd8de8a"
I20260809 16:22:56.144438 10283 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "38811d92d6db4a369472a4df011205fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fdda3347c15444659357d8e9bfce7afa"
I20260809 16:22:56.144517 10405 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6afec1683ee64fd1952734669cd8de8a [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:22:56.144515 10283 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fdda3347c15444659357d8e9bfce7afa [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:22:56.145282 10283 raft_consensus.cc:2454] T 00000000000000000000000000000000 P fdda3347c15444659357d8e9bfce7afa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38811d92d6db4a369472a4df011205fe in term 1.
I20260809 16:22:56.145351 10405 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 6afec1683ee64fd1952734669cd8de8a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38811d92d6db4a369472a4df011205fe in term 1.
I20260809 16:22:56.145450 10127 leader_election.cc:302] T 00000000000000000000000000000000 P 38811d92d6db4a369472a4df011205fe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38811d92d6db4a369472a4df011205fe, fdda3347c15444659357d8e9bfce7afa; no voters:
I20260809 16:22:56.145555 10480 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 38811d92d6db4a369472a4df011205fe [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:22:56.145669 10480 raft_consensus.cc:686] T 00000000000000000000000000000000 P 38811d92d6db4a369472a4df011205fe [term 1 LEADER]: Becoming Leader. State: Replica: 38811d92d6db4a369472a4df011205fe, State: Running, Role: LEADER
I20260809 16:22:56.145768 10480 consensus_queue.cc:237] T 00000000000000000000000000000000 P 38811d92d6db4a369472a4df011205fe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38811d92d6db4a369472a4df011205fe" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 37071 } } peers { permanent_uuid: "fdda3347c15444659357d8e9bfce7afa" member_type: VOTER last_known_addr { host: "127.21.51.189" port: 42617 } } peers { permanent_uuid: "6afec1683ee64fd1952734669cd8de8a" member_type: VOTER last_known_addr { host: "127.21.51.188" port: 39369 } }
I20260809 16:22:56.146180 10525 sys_catalog.cc:455] T 00000000000000000000000000000000 P 38811d92d6db4a369472a4df011205fe [sys.catalog]: SysCatalogTable state changed. Reason: New leader 38811d92d6db4a369472a4df011205fe. Latest consensus state: current_term: 1 leader_uuid: "38811d92d6db4a369472a4df011205fe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38811d92d6db4a369472a4df011205fe" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 37071 } } peers { permanent_uuid: "fdda3347c15444659357d8e9bfce7afa" member_type: VOTER last_known_addr { host: "127.21.51.189" port: 42617 } } peers { permanent_uuid: "6afec1683ee64fd1952734669cd8de8a" member_type: VOTER last_known_addr { host: "127.21.51.188" port: 39369 } } }
I20260809 16:22:56.146291 10525 sys_catalog.cc:458] T 00000000000000000000000000000000 P 38811d92d6db4a369472a4df011205fe [sys.catalog]: This master's current role is: LEADER
I20260809 16:22:56.146548 10531 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 16:22:56.146797 10531 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 16:22:56.148259 10405 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 6afec1683ee64fd1952734669cd8de8a [term 1 FOLLOWER]: Refusing update from remote peer 38811d92d6db4a369472a4df011205fe: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 16:22:56.148305 10283 raft_consensus.cc:1261] T 00000000000000000000000000000000 P fdda3347c15444659357d8e9bfce7afa [term 1 FOLLOWER]: Refusing update from remote peer 38811d92d6db4a369472a4df011205fe: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 16:22:56.148593 10525 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 38811d92d6db4a369472a4df011205fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "6afec1683ee64fd1952734669cd8de8a" member_type: VOTER last_known_addr { host: "127.21.51.188" port: 39369 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 16:22:56.148752 10480 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 38811d92d6db4a369472a4df011205fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "fdda3347c15444659357d8e9bfce7afa" member_type: VOTER last_known_addr { host: "127.21.51.189" port: 42617 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 16:22:56.149961 10508 sys_catalog.cc:455] T 00000000000000000000000000000000 P fdda3347c15444659357d8e9bfce7afa [sys.catalog]: SysCatalogTable state changed. Reason: New leader 38811d92d6db4a369472a4df011205fe. Latest consensus state: current_term: 1 leader_uuid: "38811d92d6db4a369472a4df011205fe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38811d92d6db4a369472a4df011205fe" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 37071 } } peers { permanent_uuid: "fdda3347c15444659357d8e9bfce7afa" member_type: VOTER last_known_addr { host: "127.21.51.189" port: 42617 } } peers { permanent_uuid: "6afec1683ee64fd1952734669cd8de8a" member_type: VOTER last_known_addr { host: "127.21.51.188" port: 39369 } } }
I20260809 16:22:56.150055 10508 sys_catalog.cc:458] T 00000000000000000000000000000000 P fdda3347c15444659357d8e9bfce7afa [sys.catalog]: This master's current role is: FOLLOWER
I20260809 16:22:56.150007 10445 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6afec1683ee64fd1952734669cd8de8a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 38811d92d6db4a369472a4df011205fe. Latest consensus state: current_term: 1 leader_uuid: "38811d92d6db4a369472a4df011205fe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38811d92d6db4a369472a4df011205fe" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 37071 } } peers { permanent_uuid: "fdda3347c15444659357d8e9bfce7afa" member_type: VOTER last_known_addr { host: "127.21.51.189" port: 42617 } } peers { permanent_uuid: "6afec1683ee64fd1952734669cd8de8a" member_type: VOTER last_known_addr { host: "127.21.51.188" port: 39369 } } }
I20260809 16:22:56.150084 10445 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6afec1683ee64fd1952734669cd8de8a [sys.catalog]: This master's current role is: FOLLOWER
I20260809 16:22:56.150857 10480 sys_catalog.cc:455] T 00000000000000000000000000000000 P 38811d92d6db4a369472a4df011205fe [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "38811d92d6db4a369472a4df011205fe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38811d92d6db4a369472a4df011205fe" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 37071 } } peers { permanent_uuid: "fdda3347c15444659357d8e9bfce7afa" member_type: VOTER last_known_addr { host: "127.21.51.189" port: 42617 } } peers { permanent_uuid: "6afec1683ee64fd1952734669cd8de8a" member_type: VOTER last_known_addr { host: "127.21.51.188" port: 39369 } } }
I20260809 16:22:56.150954 10480 sys_catalog.cc:458] T 00000000000000000000000000000000 P 38811d92d6db4a369472a4df011205fe [sys.catalog]: This master's current role is: LEADER
I20260809 16:22:56.150998 10480 sys_catalog.cc:455] T 00000000000000000000000000000000 P 38811d92d6db4a369472a4df011205fe [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "38811d92d6db4a369472a4df011205fe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38811d92d6db4a369472a4df011205fe" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 37071 } } peers { permanent_uuid: "fdda3347c15444659357d8e9bfce7afa" member_type: VOTER last_known_addr { host: "127.21.51.189" port: 42617 } } peers { permanent_uuid: "6afec1683ee64fd1952734669cd8de8a" member_type: VOTER last_known_addr { host: "127.21.51.188" port: 39369 } } }
I20260809 16:22:56.151053 10480 sys_catalog.cc:458] T 00000000000000000000000000000000 P 38811d92d6db4a369472a4df011205fe [sys.catalog]: This master's current role is: LEADER
I20260809 16:22:56.151105 10445 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6afec1683ee64fd1952734669cd8de8a [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "38811d92d6db4a369472a4df011205fe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38811d92d6db4a369472a4df011205fe" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 37071 } } peers { permanent_uuid: "fdda3347c15444659357d8e9bfce7afa" member_type: VOTER last_known_addr { host: "127.21.51.189" port: 42617 } } peers { permanent_uuid: "6afec1683ee64fd1952734669cd8de8a" member_type: VOTER last_known_addr { host: "127.21.51.188" port: 39369 } } }
I20260809 16:22:56.151202 10445 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6afec1683ee64fd1952734669cd8de8a [sys.catalog]: This master's current role is: FOLLOWER
I20260809 16:22:56.151186 10532 mvcc.cc:205] Tried to move back new op lower bound from 7316654391900610560 to 7316654391893667840. Current Snapshot: MvccSnapshot[applied={T|T < 7316654391900610560 or (T in {7316654391900610560})}]
I20260809 16:22:56.151198 10508 sys_catalog.cc:455] T 00000000000000000000000000000000 P fdda3347c15444659357d8e9bfce7afa [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "38811d92d6db4a369472a4df011205fe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38811d92d6db4a369472a4df011205fe" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 37071 } } peers { permanent_uuid: "fdda3347c15444659357d8e9bfce7afa" member_type: VOTER last_known_addr { host: "127.21.51.189" port: 42617 } } peers { permanent_uuid: "6afec1683ee64fd1952734669cd8de8a" member_type: VOTER last_known_addr { host: "127.21.51.188" port: 39369 } } }
I20260809 16:22:56.151270 10508 sys_catalog.cc:458] T 00000000000000000000000000000000 P fdda3347c15444659357d8e9bfce7afa [sys.catalog]: This master's current role is: FOLLOWER
I20260809 16:22:56.151190 10531 catalog_manager.cc:1347] Generated new cluster ID: 80be2e872f024a22a0aa3bc8b47958c8
I20260809 16:22:56.151350 10531 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 16:22:56.151376 10535 mvcc.cc:205] Tried to move back new op lower bound from 7316654391900610560 to 7316654391893667840. Current Snapshot: MvccSnapshot[applied={T|T < 7316654391900610560}]
I20260809 16:22:56.151464 10533 mvcc.cc:205] Tried to move back new op lower bound from 7316654391900610560 to 7316654391893667840. Current Snapshot: MvccSnapshot[applied={T|T < 7316654391900610560}]
I20260809 16:22:56.163563 10531 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 16:22:56.164139 10531 catalog_manager.cc:1504] Loading token signing keys...
I20260809 16:22:56.192269 10531 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 38811d92d6db4a369472a4df011205fe: Generated new TSK 0
I20260809 16:22:56.192528 10531 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 16:22:56.789984 10473 catalog_manager.cc:1259] Loaded cluster ID: 80be2e872f024a22a0aa3bc8b47958c8
I20260809 16:22:56.790092 10473 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 6afec1683ee64fd1952734669cd8de8a: loading cluster ID for follower catalog manager: success
I20260809 16:22:56.790917 10473 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 6afec1683ee64fd1952734669cd8de8a: acquiring CA information for follower catalog manager: success
I20260809 16:22:56.791230 10473 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 6afec1683ee64fd1952734669cd8de8a: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 16:22:56.851325 10522 catalog_manager.cc:1259] Loaded cluster ID: 80be2e872f024a22a0aa3bc8b47958c8
I20260809 16:22:56.851397 10522 catalog_manager.cc:1552] T 00000000000000000000000000000000 P fdda3347c15444659357d8e9bfce7afa: loading cluster ID for follower catalog manager: success
I20260809 16:22:56.852298 10522 catalog_manager.cc:1574] T 00000000000000000000000000000000 P fdda3347c15444659357d8e9bfce7afa: acquiring CA information for follower catalog manager: success
I20260809 16:22:56.852613 10522 catalog_manager.cc:1602] T 00000000000000000000000000000000 P fdda3347c15444659357d8e9bfce7afa: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 16:22:57.254011 21710 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrNqUCn/build/release/bin/kudu with pid 10109
I20260809 16:22:57.266513 21710 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrNqUCn/build/release/bin/kudu with pid 10214
I20260809 16:22:57.278205 21710 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrNqUCn/build/release/bin/kudu with pid 10335
2026-08-09T16:22:57Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (3309 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786292417722232-21710-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 16:22:57 dist-test-slave-s11t krb5kdc[10552](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 16:22:57 dist-test-slave-s11t krb5kdc[10552](info): set up 2 sockets
Aug 09 16:22:57 dist-test-slave-s11t krb5kdc[10552](info): commencing operation
krb5kdc: starting...
W20260809 16:22:58.572150 21710 mini_kdc.cc:121] Time spent starting KDC: real 1.270s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786292417722232-21710-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786292417722232-21710-0/krb5kdc/test-user.keytab.
Aug 09 16:22:58 dist-test-slave-s11t krb5kdc[10552](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292578, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T16:22:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T16:22:58Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.51.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.51.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.51.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.190.keytab.
Entry for principal kudu/127.21.51.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.190.keytab.
I20260809 16:22:58.696246 21710 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrNqUCn/build/release/bin/kudu
/tmp/dist-test-taskrNqUCn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786292417722232-21710-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786292417722232-21710-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786292417722232-21710-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786292417722232-21710-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.51.190:38079
--webserver_interface=127.21.51.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.190.keytab
--principal=kudu/127.21.51.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.51.148:32991
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.51.190:38079
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786292417722232-21710-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786292417722232-21710-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786292417722232-21710-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:22:58.779632 10569 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:22:58.779866 10569 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:22:58.779907 10569 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:22:58.781061 10569 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 16:22:58.781128 10569 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:22:58.781153 10569 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:22:58.781183 10569 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 16:22:58.781215 10569 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 16:22:58.782701 10569 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.51.148:32991
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786292417722232-21710-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786292417722232-21710-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.51.190:38079
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.190.keytab
--principal=kudu/127.21.51.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.51.190:38079
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786292417722232-21710-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.51.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10569
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786292417722232-21710-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:22:58.782979 10569 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:22:58.783226 10569 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:22:58.788220 10577 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:22:58.788228 10581 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:22:58.788195 10578 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:22:58.788539 10569 server_base.cc:1034] running on GCE node
I20260809 16:22:58.788846 10569 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:22:58.789196 10569 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:22:58.790349 10569 hybrid_clock.cc:648] HybridClock initialized: now 1786292578790330 us; error 43 us; skew 500 ppm
Aug 09 16:22:58 dist-test-slave-s11t krb5kdc[10552](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292578, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.51.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:22:58.793061 10569 init.cc:377] Logged in from keytab as kudu/127.21.51.190@KRBTEST.COM (short username kudu)
I20260809 16:22:58.793527 10569 webserver.cc:460] Webserver started at http://127.21.51.190:43803/ using document root <none> and password file <none>
I20260809 16:22:58.793793 10569 fs_manager.cc:359] Metadata directory not provided
I20260809 16:22:58.793869 10569 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:22:58.794023 10569 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:22:58.795082 10569 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786292417722232-21710-0/minicluster-data/master-0/data/instance:
uuid: "e63204d789e44d57a3309d31f246319a"
format_stamp: "Formatted at 2026-08-09 16:22:58 on dist-test-slave-s11t"
server_key: "96b6b6085d1327b409dfa5e5ea0acb9e"
server_key_iv: "8c311fa13fa692e41056ec9268736b91"
server_key_version: "encryptionkey@0"
I20260809 16:22:58.795455 10569 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786292417722232-21710-0/minicluster-data/master-0/wal/instance:
uuid: "e63204d789e44d57a3309d31f246319a"
format_stamp: "Formatted at 2026-08-09 16:22:58 on dist-test-slave-s11t"
server_key: "96b6b6085d1327b409dfa5e5ea0acb9e"
server_key_iv: "8c311fa13fa692e41056ec9268736b91"
server_key_version: "encryptionkey@0"
I20260809 16:22:58.797056 10569 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.001s
I20260809 16:22:58.797839 10591 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:22:58.798060 10569 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:22:58.798169 10569 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786292417722232-21710-0/minicluster-data/master-0/data,/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786292417722232-21710-0/minicluster-data/master-0/wal
uuid: "e63204d789e44d57a3309d31f246319a"
format_stamp: "Formatted at 2026-08-09 16:22:58 on dist-test-slave-s11t"
server_key: "96b6b6085d1327b409dfa5e5ea0acb9e"
server_key_iv: "8c311fa13fa692e41056ec9268736b91"
server_key_version: "encryptionkey@0"
I20260809 16:22:58.798245 10569 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786292417722232-21710-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786292417722232-21710-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786292417722232-21710-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:22:58.819201 10569 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:22:58.821302 10569 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:22:58.821472 10569 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:22:58.825634 10569 rpc_server.cc:307] RPC server started. Bound to: 127.21.51.190:38079
I20260809 16:22:58.825670 10673 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.51.190:38079 every 8 connection(s)
I20260809 16:22:58.826012 10569 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786292417722232-21710-0/minicluster-data/master-0/data/info.pb
I20260809 16:22:58.826872 10674 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:22:58.829272 10674 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e63204d789e44d57a3309d31f246319a: Bootstrap starting.
I20260809 16:22:58.829855 10674 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e63204d789e44d57a3309d31f246319a: Neither blocks nor log segments found. Creating new log.
I20260809 16:22:58.830158 10674 log.cc:824] T 00000000000000000000000000000000 P e63204d789e44d57a3309d31f246319a: Log is configured to *not* fsync() on all Append() calls
I20260809 16:22:58.830912 10674 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e63204d789e44d57a3309d31f246319a: No bootstrap required, opened a new log
I20260809 16:22:58.831269 21710 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrNqUCn/build/release/bin/kudu as pid 10569
I20260809 16:22:58.831380 21710 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786292417722232-21710-0/minicluster-data/master-0/wal/instance
I20260809 16:22:58.831584 21710 external_mini_cluster.cc:1428] Setting key bc9c9c2277390d9e23f58fcfc020e1b4
I20260809 16:22:58.833036 10674 raft_consensus.cc:348] T 00000000000000000000000000000000 P e63204d789e44d57a3309d31f246319a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e63204d789e44d57a3309d31f246319a" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 38079 } }
I20260809 16:22:58.833204 10674 raft_consensus.cc:374] T 00000000000000000000000000000000 P e63204d789e44d57a3309d31f246319a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:22:58.833242 10674 raft_consensus.cc:729] T 00000000000000000000000000000000 P e63204d789e44d57a3309d31f246319a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e63204d789e44d57a3309d31f246319a, State: Initialized, Role: FOLLOWER
I20260809 16:22:58.833416 10674 consensus_queue.cc:260] T 00000000000000000000000000000000 P e63204d789e44d57a3309d31f246319a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e63204d789e44d57a3309d31f246319a" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 38079 } }
I20260809 16:22:58.833513 10674 raft_consensus.cc:388] T 00000000000000000000000000000000 P e63204d789e44d57a3309d31f246319a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 16:22:58.833551 10674 raft_consensus.cc:482] T 00000000000000000000000000000000 P e63204d789e44d57a3309d31f246319a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 16:22:58.833593 10674 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e63204d789e44d57a3309d31f246319a [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:22:58.834585 10674 raft_consensus.cc:504] T 00000000000000000000000000000000 P e63204d789e44d57a3309d31f246319a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e63204d789e44d57a3309d31f246319a" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 38079 } }
I20260809 16:22:58.834672 10674 leader_election.cc:302] T 00000000000000000000000000000000 P e63204d789e44d57a3309d31f246319a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e63204d789e44d57a3309d31f246319a; no voters:
I20260809 16:22:58.834867 10674 leader_election.cc:288] T 00000000000000000000000000000000 P e63204d789e44d57a3309d31f246319a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 16:22:58.834921 10679 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e63204d789e44d57a3309d31f246319a [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:22:58.835076 10679 raft_consensus.cc:686] T 00000000000000000000000000000000 P e63204d789e44d57a3309d31f246319a [term 1 LEADER]: Becoming Leader. State: Replica: e63204d789e44d57a3309d31f246319a, State: Running, Role: LEADER
I20260809 16:22:58.835129 10674 sys_catalog.cc:564] T 00000000000000000000000000000000 P e63204d789e44d57a3309d31f246319a [sys.catalog]: configured and running, proceeding with master startup.
I20260809 16:22:58.835248 10679 consensus_queue.cc:237] T 00000000000000000000000000000000 P e63204d789e44d57a3309d31f246319a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e63204d789e44d57a3309d31f246319a" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 38079 } }
I20260809 16:22:58.835776 10682 sys_catalog.cc:455] T 00000000000000000000000000000000 P e63204d789e44d57a3309d31f246319a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e63204d789e44d57a3309d31f246319a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e63204d789e44d57a3309d31f246319a" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 38079 } } }
I20260809 16:22:58.836030 10682 sys_catalog.cc:458] T 00000000000000000000000000000000 P e63204d789e44d57a3309d31f246319a [sys.catalog]: This master's current role is: LEADER
I20260809 16:22:58.835943 10683 sys_catalog.cc:455] T 00000000000000000000000000000000 P e63204d789e44d57a3309d31f246319a [sys.catalog]: SysCatalogTable state changed. Reason: New leader e63204d789e44d57a3309d31f246319a. Latest consensus state: current_term: 1 leader_uuid: "e63204d789e44d57a3309d31f246319a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e63204d789e44d57a3309d31f246319a" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 38079 } } }
I20260809 16:22:58.836519 10695 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 16:22:58.836530 10683 sys_catalog.cc:458] T 00000000000000000000000000000000 P e63204d789e44d57a3309d31f246319a [sys.catalog]: This master's current role is: LEADER
I20260809 16:22:58.837211 10695 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 09 16:22:58 dist-test-slave-s11t krb5kdc[10552](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292578, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.51.190@KRBTEST.COM
I20260809 16:22:58.839888 10695 catalog_manager.cc:1347] Generated new cluster ID: db74b34bb2c542799a80dcceada9e52e
I20260809 16:22:58.839951 10695 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 16:22:58.843381 10678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:22:58.832234 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45806 (local address 127.21.51.190:38079)
0809 16:22:58.832405 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0809 16:22:58.832413 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 16:22:58.832440 (+ 27us) server_negotiation.cc:407] Connection header received
0809 16:22:58.833020 (+ 580us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:22:58.833027 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:22:58.833072 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:22:58.833185 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:22:58.834044 (+ 859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:22:58.835368 (+ 1324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:22:58.836598 (+ 1230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:22:58.837302 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:22:58.837412 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:22:58.839880 (+ 2468us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:22:58.839884 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:22:58.839884 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:22:58.839894 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:22:58.841755 (+ 1861us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:22:58.842137 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:22:58.842140 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:22:58.842140 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:22:58.842257 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:22:58.842489 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:22:58.842490 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:22:58.842491 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:22:58.842744 (+ 253us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:22:58.842917 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0809 16:22:58.843012 (+ 95us) server_negotiation.cc:299] Negotiation successful
0809 16:22:58.843171 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":122,"thread_start_us":84,"threads_started":1}
I20260809 16:22:58.844295 21710 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 16:22:58.859370 10695 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 16:22:58.859965 10695 catalog_manager.cc:1504] Loading token signing keys...
I20260809 16:22:58.867283 10695 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e63204d789e44d57a3309d31f246319a: Generated new TSK 0
I20260809 16:22:58.867476 10695 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 16:22:58.898499 21710 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrNqUCn/build/release/bin/kudu with pid 10569
2026-08-09T16:22:58Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (1624 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292417722232-21710-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 16:22:58 dist-test-slave-s11t krb5kdc[10712](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 16:22:58 dist-test-slave-s11t krb5kdc[10712](info): set up 2 sockets
Aug 09 16:22:58 dist-test-slave-s11t krb5kdc[10712](info): commencing operation
krb5kdc: starting...
W20260809 16:23:00.166637 21710 mini_kdc.cc:121] Time spent starting KDC: real 1.239s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292417722232-21710-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292417722232-21710-0/krb5kdc/test-user.keytab.
Aug 09 16:23:00 dist-test-slave-s11t krb5kdc[10712](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292580, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T16:23:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T16:23:00Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.51.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.51.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.51.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.190.keytab.
Entry for principal kudu/127.21.51.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.190.keytab.
I20260809 16:23:00.289783 21710 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrNqUCn/build/release/bin/kudu
/tmp/dist-test-taskrNqUCn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292417722232-21710-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292417722232-21710-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292417722232-21710-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292417722232-21710-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.51.190:36707
--webserver_interface=127.21.51.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.190.keytab
--principal=kudu/127.21.51.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.51.148:40989
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.51.190:36707
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292417722232-21710-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292417722232-21710-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292417722232-21710-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:23:00.371701 10732 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:23:00.371937 10732 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:23:00.371975 10732 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:23:00.373124 10732 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 16:23:00.373196 10732 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:23:00.373230 10732 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:23:00.373263 10732 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 16:23:00.373286 10732 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 16:23:00.374734 10732 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.51.148:40989
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292417722232-21710-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292417722232-21710-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.51.190:36707
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.190.keytab
--principal=kudu/127.21.51.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.51.190:36707
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292417722232-21710-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.51.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10732
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292417722232-21710-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:23:00.375020 10732 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:23:00.375252 10732 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:23:00.380234 10742 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:23:00.380234 10740 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:23:00.380535 10732 server_base.cc:1034] running on GCE node
W20260809 16:23:00.380242 10739 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:23:00.380897 10732 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:23:00.381199 10732 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:23:00.382378 10732 hybrid_clock.cc:648] HybridClock initialized: now 1786292580382368 us; error 31 us; skew 500 ppm
Aug 09 16:23:00 dist-test-slave-s11t krb5kdc[10712](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292580, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.51.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:23:00.384948 10732 init.cc:377] Logged in from keytab as kudu/127.21.51.190@KRBTEST.COM (short username kudu)
I20260809 16:23:00.385362 10732 webserver.cc:460] Webserver started at http://127.21.51.190:45677/ using document root <none> and password file <none>
I20260809 16:23:00.385610 10732 fs_manager.cc:359] Metadata directory not provided
I20260809 16:23:00.385679 10732 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:23:00.385811 10732 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:23:00.386817 10732 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292417722232-21710-0/minicluster-data/master-0/data/instance:
uuid: "3b5cfde595b240be93937bb3ae6a7190"
format_stamp: "Formatted at 2026-08-09 16:23:00 on dist-test-slave-s11t"
server_key: "426457e3792cbbc285f1a9de711bb57f"
server_key_iv: "a6f0c5a824caf21e65252a259d7b5892"
server_key_version: "encryptionkey@0"
I20260809 16:23:00.387177 10732 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292417722232-21710-0/minicluster-data/master-0/wal/instance:
uuid: "3b5cfde595b240be93937bb3ae6a7190"
format_stamp: "Formatted at 2026-08-09 16:23:00 on dist-test-slave-s11t"
server_key: "426457e3792cbbc285f1a9de711bb57f"
server_key_iv: "a6f0c5a824caf21e65252a259d7b5892"
server_key_version: "encryptionkey@0"
I20260809 16:23:00.388453 10732 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 16:23:00.389267 10753 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:23:00.389456 10732 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 16:23:00.389550 10732 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292417722232-21710-0/minicluster-data/master-0/data,/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292417722232-21710-0/minicluster-data/master-0/wal
uuid: "3b5cfde595b240be93937bb3ae6a7190"
format_stamp: "Formatted at 2026-08-09 16:23:00 on dist-test-slave-s11t"
server_key: "426457e3792cbbc285f1a9de711bb57f"
server_key_iv: "a6f0c5a824caf21e65252a259d7b5892"
server_key_version: "encryptionkey@0"
I20260809 16:23:00.389631 10732 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292417722232-21710-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292417722232-21710-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292417722232-21710-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:23:00.397742 10732 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:23:00.399540 10732 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:23:00.399700 10732 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:23:00.403606 10732 rpc_server.cc:307] RPC server started. Bound to: 127.21.51.190:36707
I20260809 16:23:00.403640 10827 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.51.190:36707 every 8 connection(s)
I20260809 16:23:00.403999 10732 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292417722232-21710-0/minicluster-data/master-0/data/info.pb
I20260809 16:23:00.404732 21710 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrNqUCn/build/release/bin/kudu as pid 10732
I20260809 16:23:00.404862 21710 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292417722232-21710-0/minicluster-data/master-0/wal/instance
I20260809 16:23:00.405079 21710 external_mini_cluster.cc:1428] Setting key 684e7dc9530691e8afdb83f45b319f55
I20260809 16:23:00.405160 10832 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:23:00.408479 10832 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3b5cfde595b240be93937bb3ae6a7190: Bootstrap starting.
I20260809 16:23:00.409214 10832 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3b5cfde595b240be93937bb3ae6a7190: Neither blocks nor log segments found. Creating new log.
I20260809 16:23:00.409524 10832 log.cc:824] T 00000000000000000000000000000000 P 3b5cfde595b240be93937bb3ae6a7190: Log is configured to *not* fsync() on all Append() calls
I20260809 16:23:00.410334 10832 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3b5cfde595b240be93937bb3ae6a7190: No bootstrap required, opened a new log
Aug 09 16:23:00 dist-test-slave-s11t krb5kdc[10712](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292580, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.51.190@KRBTEST.COM
I20260809 16:23:00.412758 10832 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3b5cfde595b240be93937bb3ae6a7190 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b5cfde595b240be93937bb3ae6a7190" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 36707 } }
I20260809 16:23:00.412948 10832 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3b5cfde595b240be93937bb3ae6a7190 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:23:00.412994 10832 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3b5cfde595b240be93937bb3ae6a7190 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b5cfde595b240be93937bb3ae6a7190, State: Initialized, Role: FOLLOWER
I20260809 16:23:00.413132 10832 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3b5cfde595b240be93937bb3ae6a7190 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b5cfde595b240be93937bb3ae6a7190" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 36707 } }
I20260809 16:23:00.413226 10832 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3b5cfde595b240be93937bb3ae6a7190 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 16:23:00.413271 10832 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3b5cfde595b240be93937bb3ae6a7190 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 16:23:00.413321 10832 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3b5cfde595b240be93937bb3ae6a7190 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:23:00.414062 10832 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3b5cfde595b240be93937bb3ae6a7190 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b5cfde595b240be93937bb3ae6a7190" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 36707 } }
I20260809 16:23:00.414175 10832 leader_election.cc:302] T 00000000000000000000000000000000 P 3b5cfde595b240be93937bb3ae6a7190 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3b5cfde595b240be93937bb3ae6a7190; no voters:
I20260809 16:23:00.414341 10832 leader_election.cc:288] T 00000000000000000000000000000000 P 3b5cfde595b240be93937bb3ae6a7190 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 16:23:00.414398 10840 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3b5cfde595b240be93937bb3ae6a7190 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:23:00.414569 10840 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3b5cfde595b240be93937bb3ae6a7190 [term 1 LEADER]: Becoming Leader. State: Replica: 3b5cfde595b240be93937bb3ae6a7190, State: Running, Role: LEADER
I20260809 16:23:00.414637 10832 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3b5cfde595b240be93937bb3ae6a7190 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 16:23:00.414695 10840 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3b5cfde595b240be93937bb3ae6a7190 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b5cfde595b240be93937bb3ae6a7190" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 36707 } }
I20260809 16:23:00.415316 10841 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3b5cfde595b240be93937bb3ae6a7190 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3b5cfde595b240be93937bb3ae6a7190" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b5cfde595b240be93937bb3ae6a7190" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 36707 } } }
I20260809 16:23:00.415407 10841 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3b5cfde595b240be93937bb3ae6a7190 [sys.catalog]: This master's current role is: LEADER
I20260809 16:23:00.415575 10845 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3b5cfde595b240be93937bb3ae6a7190 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3b5cfde595b240be93937bb3ae6a7190. Latest consensus state: current_term: 1 leader_uuid: "3b5cfde595b240be93937bb3ae6a7190" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b5cfde595b240be93937bb3ae6a7190" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 36707 } } }
I20260809 16:23:00.415639 10845 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3b5cfde595b240be93937bb3ae6a7190 [sys.catalog]: This master's current role is: LEADER
I20260809 16:23:00.415629 10856 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 16:23:00.416227 10856 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 16:23:00.416740 10838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:23:00.405579 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55896 (local address 127.21.51.190:36707)
0809 16:23:00.405756 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0809 16:23:00.405763 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 16:23:00.405787 (+ 24us) server_negotiation.cc:407] Connection header received
0809 16:23:00.406264 (+ 477us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:23:00.406270 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:23:00.406314 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:23:00.406424 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:23:00.407217 (+ 793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:23:00.408100 (+ 883us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:23:00.409157 (+ 1057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:23:00.409579 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:23:00.409699 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:23:00.412264 (+ 2565us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:23:00.412271 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:23:00.412271 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:23:00.412285 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:23:00.414437 (+ 2152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:23:00.414898 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:23:00.414901 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:23:00.414902 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:23:00.414996 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:23:00.415249 (+ 253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:23:00.415250 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:23:00.415250 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:23:00.415596 (+ 346us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:23:00.416245 (+ 649us) server_negotiation.cc:1035] Waiting for connection context
0809 16:23:00.416317 (+ 72us) server_negotiation.cc:299] Negotiation successful
0809 16:23:00.416487 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":127,"thread_start_us":95,"threads_started":1}
I20260809 16:23:00.417863 10856 catalog_manager.cc:1347] Generated new cluster ID: 0c67fa9120874d83b449e4f732e4ad1a
I20260809 16:23:00.417914 10856 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 16:23:00.431797 10856 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 16:23:00.432334 10856 catalog_manager.cc:1504] Loading token signing keys...
I20260809 16:23:00.441668 10856 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3b5cfde595b240be93937bb3ae6a7190: Generated new TSK 0
I20260809 16:23:00.441859 10856 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.21.51.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.51.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.51.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.129.keytab.
Entry for principal kudu/127.21.51.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.129.keytab.
I20260809 16:23:00.469691 21710 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrNqUCn/build/release/bin/kudu
/tmp/dist-test-taskrNqUCn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292417722232-21710-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292417722232-21710-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292417722232-21710-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292417722232-21710-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.51.129:0
--local_ip_for_outbound_sockets=127.21.51.129
--webserver_interface=127.21.51.129
--webserver_port=0
--tserver_master_addrs=127.21.51.190:36707
--keytab_file=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.129.keytab
--principal=kudu/127.21.51.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.51.148:40989
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292417722232-21710-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292417722232-21710-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292417722232-21710-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:23:00.552650 10871 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:23:00.552918 10871 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:23:00.552945 10871 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:23:00.554251 10871 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:23:00.554294 10871 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:23:00.554323 10871 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.51.129
I20260809 16:23:00.555922 10871 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.51.148:40989
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292417722232-21710-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292417722232-21710-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.129.keytab
--principal=kudu/127.21.51.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.51.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292417722232-21710-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.51.129
--webserver_port=0
--tserver_master_addrs=127.21.51.190:36707
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10871
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.51.129
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292417722232-21710-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:23:00.556190 10871 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:23:00.556453 10871 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:23:00.561947 10881 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:23:00.561969 10878 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:23:00.562255 10871 server_base.cc:1034] running on GCE node
W20260809 16:23:00.562007 10877 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:23:00.562523 10871 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:23:00.562768 10871 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:23:00.563949 10871 hybrid_clock.cc:648] HybridClock initialized: now 1786292580563944 us; error 39 us; skew 500 ppm
Aug 09 16:23:00 dist-test-slave-s11t krb5kdc[10712](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292580, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.51.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:23:00.570955 10871 init.cc:377] Logged in from keytab as kudu/127.21.51.129@KRBTEST.COM (short username kudu)
I20260809 16:23:00.571349 10871 webserver.cc:460] Webserver started at http://127.21.51.129:43581/ using document root <none> and password file <none>
I20260809 16:23:00.571589 10871 fs_manager.cc:359] Metadata directory not provided
I20260809 16:23:00.571637 10871 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:23:00.571791 10871 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:23:00.572898 10871 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292417722232-21710-0/minicluster-data/ts-0/data/instance:
uuid: "57576785ec93441f9b827f3a8b6df2ed"
format_stamp: "Formatted at 2026-08-09 16:23:00 on dist-test-slave-s11t"
server_key: "47e6daf2d4b2c8ffbb5ffec74976c636"
server_key_iv: "acafa1b1889fe54c341da0eec10112e2"
server_key_version: "encryptionkey@0"
I20260809 16:23:00.573242 10871 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292417722232-21710-0/minicluster-data/ts-0/wal/instance:
uuid: "57576785ec93441f9b827f3a8b6df2ed"
format_stamp: "Formatted at 2026-08-09 16:23:00 on dist-test-slave-s11t"
server_key: "47e6daf2d4b2c8ffbb5ffec74976c636"
server_key_iv: "acafa1b1889fe54c341da0eec10112e2"
server_key_version: "encryptionkey@0"
I20260809 16:23:00.574576 10871 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 16:23:00.575347 10892 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:23:00.575567 10871 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:23:00.575649 10871 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292417722232-21710-0/minicluster-data/ts-0/data,/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292417722232-21710-0/minicluster-data/ts-0/wal
uuid: "57576785ec93441f9b827f3a8b6df2ed"
format_stamp: "Formatted at 2026-08-09 16:23:00 on dist-test-slave-s11t"
server_key: "47e6daf2d4b2c8ffbb5ffec74976c636"
server_key_iv: "acafa1b1889fe54c341da0eec10112e2"
server_key_version: "encryptionkey@0"
I20260809 16:23:00.575742 10871 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292417722232-21710-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292417722232-21710-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292417722232-21710-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:23:00.593716 10871 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:23:00.596046 10871 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:23:00.596200 10871 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:23:00.596453 10871 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:23:00.596781 10871 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:23:00.596828 10871 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:23:00.596850 10871 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:23:00.596863 10871 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:23:00.602782 10871 rpc_server.cc:307] RPC server started. Bound to: 127.21.51.129:41875
I20260809 16:23:00.602849 11064 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.51.129:41875 every 8 connection(s)
I20260809 16:23:00.603142 10871 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292417722232-21710-0/minicluster-data/ts-0/data/info.pb
I20260809 16:23:00.604921 21710 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrNqUCn/build/release/bin/kudu as pid 10871
I20260809 16:23:00.605041 21710 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292417722232-21710-0/minicluster-data/ts-0/wal/instance
I20260809 16:23:00.605265 21710 external_mini_cluster.cc:1428] Setting key 6dccf0d8fe98e2d59175d4ed635cec1c
Aug 09 16:23:00 dist-test-slave-s11t krb5kdc[10712](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292580, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.51.129@KRBTEST.COM for kudu/127.21.51.190@KRBTEST.COM
I20260809 16:23:00.613284 10838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:23:00.603864 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.51.129:54521 (local address 127.21.51.190:36707)
0809 16:23:00.603910 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0809 16:23:00.603912 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:23:00.604164 (+ 252us) server_negotiation.cc:407] Connection header received
0809 16:23:00.604723 (+ 559us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:23:00.604724 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:23:00.604738 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:23:00.604795 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:23:00.606054 (+ 1259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:23:00.606882 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:23:00.607932 (+ 1050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:23:00.608150 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:23:00.608230 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:23:00.610373 (+ 2143us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:23:00.610375 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:23:00.610375 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:23:00.610383 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:23:00.611788 (+ 1405us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:23:00.612228 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:23:00.612235 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:23:00.612235 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:23:00.612298 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:23:00.612567 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:23:00.612567 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:23:00.612568 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:23:00.612732 (+ 164us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:23:00.612795 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0809 16:23:00.613094 (+ 299us) server_negotiation.cc:299] Negotiation successful
0809 16:23:00.613190 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260809 16:23:00.613468 11067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:23:00.603859 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:36707 (local address 127.21.51.129:54521)
0809 16:23:00.604043 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0809 16:23:00.604079 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0809 16:23:00.604586 (+ 507us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:23:00.604892 (+ 306us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:23:00.604896 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:23:00.605633 (+ 737us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:23:00.605922 (+ 289us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:23:00.605928 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:23:00.606991 (+ 1063us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:23:00.606992 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:23:00.607841 (+ 849us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:23:00.607844 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:23:00.608238 (+ 394us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:23:00.608239 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:23:00.608384 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:23:00.608918 (+ 534us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:23:00.608926 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:23:00.610269 (+ 1343us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:23:00.611895 (+ 1626us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:23:00.611903 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:23:00.611905 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:23:00.612138 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:23:00.612372 (+ 234us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:23:00.612373 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:23:00.612373 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:23:00.612490 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:23:00.612808 (+ 318us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:23:00.612815 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:23:00.612954 (+ 139us) client_negotiation.cc:769] Sending connection context
0809 16:23:00.613060 (+ 106us) client_negotiation.cc:240] Negotiation successful
0809 16:23:00.613220 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":98,"threads_started":1}
I20260809 16:23:00.613987 11065 heartbeater.cc:347] Connected to a master server at 127.21.51.190:36707
I20260809 16:23:00.614108 11065 heartbeater.cc:464] Registering TS with master...
I20260809 16:23:00.614283 11065 heartbeater.cc:511] Master 127.21.51.190:36707 requested a full tablet report, sending...
I20260809 16:23:00.614852 10777 ts_manager.cc:194] Registered new tserver with Master: 57576785ec93441f9b827f3a8b6df2ed (127.21.51.129:41875)
WARNING: no policy specified for kudu/127.21.51.130@KRBTEST.COM; defaulting to no policy
I20260809 16:23:00.615594 10777 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.51.129@KRBTEST.COM'} at 127.21.51.129:54521
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.51.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.51.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.130.keytab.
Entry for principal kudu/127.21.51.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.130.keytab.
I20260809 16:23:00.633360 21710 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrNqUCn/build/release/bin/kudu
/tmp/dist-test-taskrNqUCn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292417722232-21710-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292417722232-21710-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292417722232-21710-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292417722232-21710-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.51.130:0
--local_ip_for_outbound_sockets=127.21.51.130
--webserver_interface=127.21.51.130
--webserver_port=0
--tserver_master_addrs=127.21.51.190:36707
--keytab_file=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.130.keytab
--principal=kudu/127.21.51.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.51.148:40989
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292417722232-21710-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292417722232-21710-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292417722232-21710-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:23:00.724247 11072 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:23:00.724463 11072 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:23:00.724488 11072 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:23:00.725780 11072 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:23:00.725828 11072 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:23:00.725857 11072 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.51.130
I20260809 16:23:00.727619 11072 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.51.148:40989
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292417722232-21710-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292417722232-21710-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.130.keytab
--principal=kudu/127.21.51.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.51.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292417722232-21710-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.51.130
--webserver_port=0
--tserver_master_addrs=127.21.51.190:36707
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11072
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.51.130
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292417722232-21710-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:23:00.727875 11072 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:23:00.728149 11072 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 16:23:00.733549 11072 server_base.cc:1034] running on GCE node
W20260809 16:23:00.733474 11080 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:23:00.733513 11085 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:23:00.733485 11081 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:23:00.734059 11072 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:23:00.734313 11072 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:23:00.735455 11072 hybrid_clock.cc:648] HybridClock initialized: now 1786292580735442 us; error 32 us; skew 500 ppm
Aug 09 16:23:00 dist-test-slave-s11t krb5kdc[10712](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292580, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.51.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:23:00.738193 11072 init.cc:377] Logged in from keytab as kudu/127.21.51.130@KRBTEST.COM (short username kudu)
I20260809 16:23:00.738786 11072 webserver.cc:460] Webserver started at http://127.21.51.130:45661/ using document root <none> and password file <none>
I20260809 16:23:00.739054 11072 fs_manager.cc:359] Metadata directory not provided
I20260809 16:23:00.739131 11072 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:23:00.739269 11072 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:23:00.740617 11072 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292417722232-21710-0/minicluster-data/ts-1/data/instance:
uuid: "5ba1dc75657f4418b6903fa6ceed3d8b"
format_stamp: "Formatted at 2026-08-09 16:23:00 on dist-test-slave-s11t"
server_key: "b1aee06ad35f0c32fe8b3fc4df3cfac2"
server_key_iv: "42725d4f5393d5976a7fc58d89e13374"
server_key_version: "encryptionkey@0"
I20260809 16:23:00.741094 11072 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292417722232-21710-0/minicluster-data/ts-1/wal/instance:
uuid: "5ba1dc75657f4418b6903fa6ceed3d8b"
format_stamp: "Formatted at 2026-08-09 16:23:00 on dist-test-slave-s11t"
server_key: "b1aee06ad35f0c32fe8b3fc4df3cfac2"
server_key_iv: "42725d4f5393d5976a7fc58d89e13374"
server_key_version: "encryptionkey@0"
I20260809 16:23:00.742625 11072 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 16:23:00.743462 11095 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:23:00.743666 11072 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:23:00.743769 11072 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292417722232-21710-0/minicluster-data/ts-1/data,/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292417722232-21710-0/minicluster-data/ts-1/wal
uuid: "5ba1dc75657f4418b6903fa6ceed3d8b"
format_stamp: "Formatted at 2026-08-09 16:23:00 on dist-test-slave-s11t"
server_key: "b1aee06ad35f0c32fe8b3fc4df3cfac2"
server_key_iv: "42725d4f5393d5976a7fc58d89e13374"
server_key_version: "encryptionkey@0"
I20260809 16:23:00.743857 11072 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292417722232-21710-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292417722232-21710-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292417722232-21710-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:23:00.757810 11072 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:23:00.760076 11072 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:23:00.760233 11072 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:23:00.760468 11072 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:23:00.760845 11072 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:23:00.760900 11072 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:23:00.760941 11072 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:23:00.760968 11072 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:23:00.766970 11072 rpc_server.cc:307] RPC server started. Bound to: 127.21.51.130:46397
I20260809 16:23:00.767006 11268 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.51.130:46397 every 8 connection(s)
I20260809 16:23:00.767334 11072 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292417722232-21710-0/minicluster-data/ts-1/data/info.pb
I20260809 16:23:00.768734 21710 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrNqUCn/build/release/bin/kudu as pid 11072
I20260809 16:23:00.768869 21710 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292417722232-21710-0/minicluster-data/ts-1/wal/instance
I20260809 16:23:00.769104 21710 external_mini_cluster.cc:1428] Setting key 9b84ca40f9752618d4a115eef516d0e8
Aug 09 16:23:00 dist-test-slave-s11t krb5kdc[10712](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292580, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.51.130@KRBTEST.COM for kudu/127.21.51.190@KRBTEST.COM
I20260809 16:23:00.778600 10838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:23:00.768089 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.51.130:58927 (local address 127.21.51.190:36707)
0809 16:23:00.768130 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0809 16:23:00.768132 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:23:00.768330 (+ 198us) server_negotiation.cc:407] Connection header received
0809 16:23:00.768905 (+ 575us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:23:00.768907 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:23:00.768920 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:23:00.768990 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:23:00.770335 (+ 1345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:23:00.770861 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:23:00.772249 (+ 1388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:23:00.772572 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:23:00.772655 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:23:00.775202 (+ 2547us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:23:00.775204 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:23:00.775204 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:23:00.775211 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:23:00.776920 (+ 1709us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:23:00.777427 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:23:00.777428 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:23:00.777428 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:23:00.777491 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:23:00.777828 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:23:00.777829 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:23:00.777829 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:23:00.777958 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:23:00.778030 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0809 16:23:00.778433 (+ 403us) server_negotiation.cc:299] Negotiation successful
0809 16:23:00.778530 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260809 16:23:00.778864 11277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:23:00.768016 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:36707 (local address 127.21.51.130:58927)
0809 16:23:00.768207 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0809 16:23:00.768241 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0809 16:23:00.768768 (+ 527us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:23:00.769127 (+ 359us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:23:00.769131 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:23:00.769945 (+ 814us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:23:00.770212 (+ 267us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:23:00.770218 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:23:00.770954 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:23:00.770955 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:23:00.772159 (+ 1204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:23:00.772161 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:23:00.772664 (+ 503us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:23:00.772665 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:23:00.772839 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:23:00.773324 (+ 485us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:23:00.773332 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:23:00.775103 (+ 1771us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:23:00.777008 (+ 1905us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:23:00.777016 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:23:00.777018 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:23:00.777337 (+ 319us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:23:00.777571 (+ 234us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:23:00.777572 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:23:00.777572 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:23:00.777723 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:23:00.778045 (+ 322us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:23:00.778052 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:23:00.778218 (+ 166us) client_negotiation.cc:769] Sending connection context
0809 16:23:00.778410 (+ 192us) client_negotiation.cc:240] Negotiation successful
0809 16:23:00.778597 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":71,"threads_started":1}
WARNING: no policy specified for kudu/127.21.51.131@KRBTEST.COM; defaulting to no policy
I20260809 16:23:00.779485 11273 heartbeater.cc:347] Connected to a master server at 127.21.51.190:36707
I20260809 16:23:00.779625 11273 heartbeater.cc:464] Registering TS with master...
I20260809 16:23:00.779862 11273 heartbeater.cc:511] Master 127.21.51.190:36707 requested a full tablet report, sending...
I20260809 16:23:00.780339 10777 ts_manager.cc:194] Registered new tserver with Master: 5ba1dc75657f4418b6903fa6ceed3d8b (127.21.51.130:46397)
I20260809 16:23:00.780809 10777 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.51.130@KRBTEST.COM'} at 127.21.51.130:58927
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.51.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.51.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.131.keytab.
Entry for principal kudu/127.21.51.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.131.keytab.
I20260809 16:23:00.797672 21710 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrNqUCn/build/release/bin/kudu
/tmp/dist-test-taskrNqUCn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292417722232-21710-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292417722232-21710-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292417722232-21710-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292417722232-21710-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.51.131:0
--local_ip_for_outbound_sockets=127.21.51.131
--webserver_interface=127.21.51.131
--webserver_port=0
--tserver_master_addrs=127.21.51.190:36707
--keytab_file=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.131.keytab
--principal=kudu/127.21.51.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.51.148:40989
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292417722232-21710-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292417722232-21710-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292417722232-21710-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:23:00.879158 11282 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:23:00.879381 11282 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:23:00.879420 11282 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:23:00.880532 11282 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:23:00.880600 11282 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:23:00.880646 11282 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.51.131
I20260809 16:23:00.882124 11282 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.51.148:40989
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292417722232-21710-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292417722232-21710-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.131.keytab
--principal=kudu/127.21.51.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.51.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292417722232-21710-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.51.131
--webserver_port=0
--tserver_master_addrs=127.21.51.190:36707
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11282
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.51.131
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292417722232-21710-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:23:00.882397 11282 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:23:00.882628 11282 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:23:00.887905 11294 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:23:00.887928 11296 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:23:00.888052 11282 server_base.cc:1034] running on GCE node
W20260809 16:23:00.887928 11293 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:23:00.888360 11282 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:23:00.888631 11282 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:23:00.889837 11282 hybrid_clock.cc:648] HybridClock initialized: now 1786292580889820 us; error 39 us; skew 500 ppm
Aug 09 16:23:00 dist-test-slave-s11t krb5kdc[10712](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292580, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.51.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:23:00.892604 11282 init.cc:377] Logged in from keytab as kudu/127.21.51.131@KRBTEST.COM (short username kudu)
I20260809 16:23:00.893115 11282 webserver.cc:460] Webserver started at http://127.21.51.131:38313/ using document root <none> and password file <none>
I20260809 16:23:00.893365 11282 fs_manager.cc:359] Metadata directory not provided
I20260809 16:23:00.893437 11282 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:23:00.893586 11282 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:23:00.894608 11282 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292417722232-21710-0/minicluster-data/ts-2/data/instance:
uuid: "274512f1889a4254b485fef854967321"
format_stamp: "Formatted at 2026-08-09 16:23:00 on dist-test-slave-s11t"
server_key: "5c0923b2d7f6c75f47c908daa014db10"
server_key_iv: "9d15cee54955387cc133adf55e93d3fc"
server_key_version: "encryptionkey@0"
I20260809 16:23:00.894954 11282 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292417722232-21710-0/minicluster-data/ts-2/wal/instance:
uuid: "274512f1889a4254b485fef854967321"
format_stamp: "Formatted at 2026-08-09 16:23:00 on dist-test-slave-s11t"
server_key: "5c0923b2d7f6c75f47c908daa014db10"
server_key_iv: "9d15cee54955387cc133adf55e93d3fc"
server_key_version: "encryptionkey@0"
I20260809 16:23:00.896288 11282 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 16:23:00.897073 11304 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:23:00.897255 11282 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 16:23:00.897362 11282 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292417722232-21710-0/minicluster-data/ts-2/data,/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292417722232-21710-0/minicluster-data/ts-2/wal
uuid: "274512f1889a4254b485fef854967321"
format_stamp: "Formatted at 2026-08-09 16:23:00 on dist-test-slave-s11t"
server_key: "5c0923b2d7f6c75f47c908daa014db10"
server_key_iv: "9d15cee54955387cc133adf55e93d3fc"
server_key_version: "encryptionkey@0"
I20260809 16:23:00.897435 11282 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292417722232-21710-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292417722232-21710-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292417722232-21710-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:23:00.931073 11282 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:23:00.933166 11282 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:23:00.933348 11282 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:23:00.933630 11282 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:23:00.933993 11282 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:23:00.934051 11282 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:23:00.934096 11282 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:23:00.934125 11282 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:23:00.939814 11282 rpc_server.cc:307] RPC server started. Bound to: 127.21.51.131:42799
I20260809 16:23:00.939916 11471 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.51.131:42799 every 8 connection(s)
I20260809 16:23:00.940227 11282 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292417722232-21710-0/minicluster-data/ts-2/data/info.pb
I20260809 16:23:00.943153 21710 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrNqUCn/build/release/bin/kudu as pid 11282
I20260809 16:23:00.943264 21710 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292417722232-21710-0/minicluster-data/ts-2/wal/instance
I20260809 16:23:00.943449 21710 external_mini_cluster.cc:1428] Setting key 76230998fddced756de322f08a3ef13a
Aug 09 16:23:00 dist-test-slave-s11t krb5kdc[10712](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292580, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.51.131@KRBTEST.COM for kudu/127.21.51.190@KRBTEST.COM
I20260809 16:23:00.950309 10838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:23:00.941033 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.51.131:59103 (local address 127.21.51.190:36707)
0809 16:23:00.941108 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0809 16:23:00.941110 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:23:00.941269 (+ 159us) server_negotiation.cc:407] Connection header received
0809 16:23:00.941790 (+ 521us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:23:00.941791 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:23:00.941824 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:23:00.941906 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:23:00.942839 (+ 933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:23:00.943635 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:23:00.944834 (+ 1199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:23:00.945050 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:23:00.945139 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:23:00.947398 (+ 2259us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:23:00.947399 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:23:00.947400 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:23:00.947408 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:23:00.948901 (+ 1493us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:23:00.949328 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:23:00.949329 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:23:00.949329 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:23:00.949400 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:23:00.949655 (+ 255us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:23:00.949656 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:23:00.949656 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:23:00.949773 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:23:00.949853 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0809 16:23:00.950130 (+ 277us) server_negotiation.cc:299] Negotiation successful
0809 16:23:00.950213 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260809 16:23:00.950507 11477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:23:00.940987 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:36707 (local address 127.21.51.131:59103)
0809 16:23:00.941141 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0809 16:23:00.941182 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0809 16:23:00.941662 (+ 480us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:23:00.941973 (+ 311us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:23:00.941977 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:23:00.942535 (+ 558us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:23:00.942747 (+ 212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:23:00.942752 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:23:00.943747 (+ 995us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:23:00.943748 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:23:00.944749 (+ 1001us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:23:00.944751 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:23:00.945151 (+ 400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:23:00.945152 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:23:00.945291 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:23:00.945826 (+ 535us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:23:00.945834 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:23:00.947292 (+ 1458us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:23:00.949002 (+ 1710us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:23:00.949008 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:23:00.949010 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:23:00.949236 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:23:00.949468 (+ 232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:23:00.949469 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:23:00.949469 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:23:00.949583 (+ 114us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:23:00.949843 (+ 260us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:23:00.949849 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:23:00.949999 (+ 150us) client_negotiation.cc:769] Sending connection context
0809 16:23:00.950108 (+ 109us) client_negotiation.cc:240] Negotiation successful
0809 16:23:00.950268 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":83,"threads_started":1}
I20260809 16:23:00.951043 11475 heartbeater.cc:347] Connected to a master server at 127.21.51.190:36707
I20260809 16:23:00.951143 11475 heartbeater.cc:464] Registering TS with master...
I20260809 16:23:00.951319 11475 heartbeater.cc:511] Master 127.21.51.190:36707 requested a full tablet report, sending...
I20260809 16:23:00.951718 10777 ts_manager.cc:194] Registered new tserver with Master: 274512f1889a4254b485fef854967321 (127.21.51.131:42799)
I20260809 16:23:00.952118 10777 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.51.131@KRBTEST.COM'} at 127.21.51.131:59103
I20260809 16:23:00.955205 21710 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 16:23:00.962531 10838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:23:00.956524 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55900 (local address 127.21.51.190:36707)
0809 16:23:00.956590 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0809 16:23:00.956593 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 16:23:00.956706 (+ 113us) server_negotiation.cc:407] Connection header received
0809 16:23:00.956779 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:23:00.956780 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:23:00.956790 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:23:00.956839 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:23:00.957420 (+ 581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:23:00.957927 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:23:00.958705 (+ 778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:23:00.958916 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:23:00.958998 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:23:00.960116 (+ 1118us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:23:00.960117 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:23:00.960117 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:23:00.960125 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:23:00.961308 (+ 1183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:23:00.961675 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:23:00.961677 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:23:00.961677 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:23:00.961736 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:23:00.961975 (+ 239us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:23:00.961976 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:23:00.961976 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:23:00.962121 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:23:00.962229 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0809 16:23:00.962360 (+ 131us) server_negotiation.cc:299] Negotiation successful
0809 16:23:00.962438 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260809 16:23:00.964226 10777 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55900:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 16:23:00.965914 10777 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 16:23:00.972741 11493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:23:00.968581 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.129:41875 (local address 127.0.0.1:49782)
0809 16:23:00.969110 (+ 529us) negotiation.cc:107] Waiting for socket to connect
0809 16:23:00.969121 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 16:23:00.969181 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:23:00.969511 (+ 330us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:23:00.969512 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:23:00.969514 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:23:00.969567 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:23:00.969568 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:23:00.970461 (+ 893us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:23:00.970462 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:23:00.971442 (+ 980us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:23:00.971445 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:23:00.972378 (+ 933us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:23:00.972379 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:23:00.972470 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:23:00.972485 (+ 15us) client_negotiation.cc:769] Sending connection context
0809 16:23:00.972572 (+ 87us) client_negotiation.cc:240] Negotiation successful
0809 16:23:00.972632 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":508,"mutex_wait_us":34,"thread_start_us":72,"threads_started":1}
I20260809 16:23:00.972769 11500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:23:00.968914 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49782 (local address 127.21.51.129:41875)
0809 16:23:00.969105 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0809 16:23:00.969109 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 16:23:00.969292 (+ 183us) server_negotiation.cc:407] Connection header received
0809 16:23:00.969338 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:23:00.969340 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:23:00.969356 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:23:00.969431 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:23:00.969643 (+ 212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:23:00.970358 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:23:00.971545 (+ 1187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:23:00.972282 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:23:00.972362 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:23:00.972431 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0809 16:23:00.972606 (+ 175us) server_negotiation.cc:299] Negotiation successful
0809 16:23:00.972663 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":156,"thread_start_us":94,"threads_started":1}
I20260809 16:23:00.972983 11497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:23:00.968363 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.131:42799 (local address 127.0.0.1:57110)
0809 16:23:00.968717 (+ 354us) negotiation.cc:107] Waiting for socket to connect
0809 16:23:00.968729 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 16:23:00.968796 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:23:00.968992 (+ 196us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:23:00.968996 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:23:00.968999 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:23:00.969070 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:23:00.969072 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:23:00.970391 (+ 1319us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:23:00.970392 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:23:00.971854 (+ 1462us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:23:00.971856 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:23:00.972685 (+ 829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:23:00.972686 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:23:00.972780 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:23:00.972792 (+ 12us) client_negotiation.cc:769] Sending connection context
0809 16:23:00.972852 (+ 60us) client_negotiation.cc:240] Negotiation successful
0809 16:23:00.972903 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":310,"mutex_wait_us":173,"thread_start_us":81,"threads_started":1}
I20260809 16:23:00.973135 11490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:23:00.968359 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.130:46397 (local address 127.0.0.1:60972)
0809 16:23:00.968478 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0809 16:23:00.968519 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0809 16:23:00.968584 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:23:00.969355 (+ 771us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:23:00.969356 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:23:00.969357 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:23:00.969409 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:23:00.969411 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:23:00.970576 (+ 1165us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:23:00.970577 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:23:00.971732 (+ 1155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:23:00.971735 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:23:00.972693 (+ 958us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:23:00.972695 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:23:00.972808 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:23:00.972814 (+ 6us) client_negotiation.cc:769] Sending connection context
0809 16:23:00.972850 (+ 36us) client_negotiation.cc:240] Negotiation successful
0809 16:23:00.972904 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":94,"mutex_wait_us":41,"thread_start_us":73,"threads_started":1}
I20260809 16:23:00.973220 11496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:23:00.968452 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57110 (local address 127.21.51.131:42799)
0809 16:23:00.968629 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0809 16:23:00.968634 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 16:23:00.968809 (+ 175us) server_negotiation.cc:407] Connection header received
0809 16:23:00.968866 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:23:00.968868 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:23:00.968887 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:23:00.968976 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:23:00.969160 (+ 184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:23:00.970276 (+ 1116us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:23:00.971928 (+ 1652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:23:00.972534 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:23:00.972597 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:23:00.972650 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0809 16:23:00.973075 (+ 425us) server_negotiation.cc:299] Negotiation successful
0809 16:23:00.973134 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":140,"thread_start_us":99,"threads_started":1}
I20260809 16:23:00.973510 11499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:23:00.968771 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60972 (local address 127.21.51.130:46397)
0809 16:23:00.968929 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0809 16:23:00.968934 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 16:23:00.969177 (+ 243us) server_negotiation.cc:407] Connection header received
0809 16:23:00.969227 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:23:00.969230 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:23:00.969246 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:23:00.969352 (+ 106us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:23:00.969471 (+ 119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:23:00.970483 (+ 1012us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:23:00.971821 (+ 1338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:23:00.972590 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:23:00.973298 (+ 708us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:23:00.973363 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0809 16:23:00.973389 (+ 26us) server_negotiation.cc:299] Negotiation successful
0809 16:23:00.973430 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":92,"threads_started":1}
I20260809 16:23:00.973659 10965 tablet_service.cc:1467] Processing CreateTablet for tablet d01444e43d0d4a639f0655acae1c1b44 (DEFAULT_TABLE table=test-table [id=6eed8c27546f492ea3609280d409f421]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:23:00.974005 10965 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d01444e43d0d4a639f0655acae1c1b44. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:23:00.974164 11158 tablet_service.cc:1467] Processing CreateTablet for tablet d01444e43d0d4a639f0655acae1c1b44 (DEFAULT_TABLE table=test-table [id=6eed8c27546f492ea3609280d409f421]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:23:00.974516 11158 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d01444e43d0d4a639f0655acae1c1b44. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:23:00.976567 11501 tablet_bootstrap.cc:492] T d01444e43d0d4a639f0655acae1c1b44 P 57576785ec93441f9b827f3a8b6df2ed: Bootstrap starting.
I20260809 16:23:00.977252 11503 tablet_bootstrap.cc:492] T d01444e43d0d4a639f0655acae1c1b44 P 5ba1dc75657f4418b6903fa6ceed3d8b: Bootstrap starting.
I20260809 16:23:00.977296 11501 tablet_bootstrap.cc:654] T d01444e43d0d4a639f0655acae1c1b44 P 57576785ec93441f9b827f3a8b6df2ed: Neither blocks nor log segments found. Creating new log.
I20260809 16:23:00.977187 11374 tablet_service.cc:1467] Processing CreateTablet for tablet d01444e43d0d4a639f0655acae1c1b44 (DEFAULT_TABLE table=test-table [id=6eed8c27546f492ea3609280d409f421]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:23:00.977531 11374 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d01444e43d0d4a639f0655acae1c1b44. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:23:00.977909 11501 log.cc:824] T d01444e43d0d4a639f0655acae1c1b44 P 57576785ec93441f9b827f3a8b6df2ed: Log is configured to *not* fsync() on all Append() calls
I20260809 16:23:00.977946 11503 tablet_bootstrap.cc:654] T d01444e43d0d4a639f0655acae1c1b44 P 5ba1dc75657f4418b6903fa6ceed3d8b: Neither blocks nor log segments found. Creating new log.
I20260809 16:23:00.978533 11503 log.cc:824] T d01444e43d0d4a639f0655acae1c1b44 P 5ba1dc75657f4418b6903fa6ceed3d8b: Log is configured to *not* fsync() on all Append() calls
I20260809 16:23:00.978796 11501 tablet_bootstrap.cc:492] T d01444e43d0d4a639f0655acae1c1b44 P 57576785ec93441f9b827f3a8b6df2ed: No bootstrap required, opened a new log
I20260809 16:23:00.978885 11501 ts_tablet_manager.cc:1397] T d01444e43d0d4a639f0655acae1c1b44 P 57576785ec93441f9b827f3a8b6df2ed: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 16:23:00.979336 11503 tablet_bootstrap.cc:492] T d01444e43d0d4a639f0655acae1c1b44 P 5ba1dc75657f4418b6903fa6ceed3d8b: No bootstrap required, opened a new log
I20260809 16:23:00.979424 11503 ts_tablet_manager.cc:1397] T d01444e43d0d4a639f0655acae1c1b44 P 5ba1dc75657f4418b6903fa6ceed3d8b: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 16:23:00.981045 11503 raft_consensus.cc:348] T d01444e43d0d4a639f0655acae1c1b44 P 5ba1dc75657f4418b6903fa6ceed3d8b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ba1dc75657f4418b6903fa6ceed3d8b" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 46397 } } peers { permanent_uuid: "274512f1889a4254b485fef854967321" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 42799 } } peers { permanent_uuid: "57576785ec93441f9b827f3a8b6df2ed" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 41875 } }
I20260809 16:23:00.981083 11501 raft_consensus.cc:348] T d01444e43d0d4a639f0655acae1c1b44 P 57576785ec93441f9b827f3a8b6df2ed [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ba1dc75657f4418b6903fa6ceed3d8b" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 46397 } } peers { permanent_uuid: "274512f1889a4254b485fef854967321" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 42799 } } peers { permanent_uuid: "57576785ec93441f9b827f3a8b6df2ed" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 41875 } }
I20260809 16:23:00.981211 11501 raft_consensus.cc:374] T d01444e43d0d4a639f0655acae1c1b44 P 57576785ec93441f9b827f3a8b6df2ed [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:23:00.981215 11503 raft_consensus.cc:374] T d01444e43d0d4a639f0655acae1c1b44 P 5ba1dc75657f4418b6903fa6ceed3d8b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:23:00.981254 11503 raft_consensus.cc:729] T d01444e43d0d4a639f0655acae1c1b44 P 5ba1dc75657f4418b6903fa6ceed3d8b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5ba1dc75657f4418b6903fa6ceed3d8b, State: Initialized, Role: FOLLOWER
I20260809 16:23:00.981232 11501 raft_consensus.cc:729] T d01444e43d0d4a639f0655acae1c1b44 P 57576785ec93441f9b827f3a8b6df2ed [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57576785ec93441f9b827f3a8b6df2ed, State: Initialized, Role: FOLLOWER
I20260809 16:23:00.981372 11501 consensus_queue.cc:260] T d01444e43d0d4a639f0655acae1c1b44 P 57576785ec93441f9b827f3a8b6df2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ba1dc75657f4418b6903fa6ceed3d8b" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 46397 } } peers { permanent_uuid: "274512f1889a4254b485fef854967321" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 42799 } } peers { permanent_uuid: "57576785ec93441f9b827f3a8b6df2ed" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 41875 } }
I20260809 16:23:00.981372 11503 consensus_queue.cc:260] T d01444e43d0d4a639f0655acae1c1b44 P 5ba1dc75657f4418b6903fa6ceed3d8b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ba1dc75657f4418b6903fa6ceed3d8b" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 46397 } } peers { permanent_uuid: "274512f1889a4254b485fef854967321" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 42799 } } peers { permanent_uuid: "57576785ec93441f9b827f3a8b6df2ed" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 41875 } }
I20260809 16:23:00.981667 11501 ts_tablet_manager.cc:1428] T d01444e43d0d4a639f0655acae1c1b44 P 57576785ec93441f9b827f3a8b6df2ed: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 16:23:00.981684 11503 ts_tablet_manager.cc:1428] T d01444e43d0d4a639f0655acae1c1b44 P 5ba1dc75657f4418b6903fa6ceed3d8b: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260809 16:23:00.981709 11065 heartbeater.cc:503] Master 127.21.51.190:36707 was elected leader, sending a full tablet report...
I20260809 16:23:00.981859 11506 tablet_bootstrap.cc:492] T d01444e43d0d4a639f0655acae1c1b44 P 274512f1889a4254b485fef854967321: Bootstrap starting.
I20260809 16:23:00.982054 11273 heartbeater.cc:503] Master 127.21.51.190:36707 was elected leader, sending a full tablet report...
I20260809 16:23:00.982529 11506 tablet_bootstrap.cc:654] T d01444e43d0d4a639f0655acae1c1b44 P 274512f1889a4254b485fef854967321: Neither blocks nor log segments found. Creating new log.
I20260809 16:23:00.983132 11506 log.cc:824] T d01444e43d0d4a639f0655acae1c1b44 P 274512f1889a4254b485fef854967321: Log is configured to *not* fsync() on all Append() calls
I20260809 16:23:00.983760 11506 tablet_bootstrap.cc:492] T d01444e43d0d4a639f0655acae1c1b44 P 274512f1889a4254b485fef854967321: No bootstrap required, opened a new log
I20260809 16:23:00.983814 11506 ts_tablet_manager.cc:1397] T d01444e43d0d4a639f0655acae1c1b44 P 274512f1889a4254b485fef854967321: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 16:23:00.985324 11506 raft_consensus.cc:348] T d01444e43d0d4a639f0655acae1c1b44 P 274512f1889a4254b485fef854967321 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ba1dc75657f4418b6903fa6ceed3d8b" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 46397 } } peers { permanent_uuid: "274512f1889a4254b485fef854967321" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 42799 } } peers { permanent_uuid: "57576785ec93441f9b827f3a8b6df2ed" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 41875 } }
I20260809 16:23:00.985451 11506 raft_consensus.cc:374] T d01444e43d0d4a639f0655acae1c1b44 P 274512f1889a4254b485fef854967321 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:23:00.985474 11506 raft_consensus.cc:729] T d01444e43d0d4a639f0655acae1c1b44 P 274512f1889a4254b485fef854967321 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 274512f1889a4254b485fef854967321, State: Initialized, Role: FOLLOWER
I20260809 16:23:00.985548 11506 consensus_queue.cc:260] T d01444e43d0d4a639f0655acae1c1b44 P 274512f1889a4254b485fef854967321 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ba1dc75657f4418b6903fa6ceed3d8b" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 46397 } } peers { permanent_uuid: "274512f1889a4254b485fef854967321" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 42799 } } peers { permanent_uuid: "57576785ec93441f9b827f3a8b6df2ed" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 41875 } }
I20260809 16:23:00.985751 11506 ts_tablet_manager.cc:1428] T d01444e43d0d4a639f0655acae1c1b44 P 274512f1889a4254b485fef854967321: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260809 16:23:00.985819 11475 heartbeater.cc:503] Master 127.21.51.190:36707 was elected leader, sending a full tablet report...
W20260809 16:23:01.017932 11274 tablet.cc:2369] T d01444e43d0d4a639f0655acae1c1b44 P 5ba1dc75657f4418b6903fa6ceed3d8b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 16:23:01.098932 11508 raft_consensus.cc:482] T d01444e43d0d4a639f0655acae1c1b44 P 57576785ec93441f9b827f3a8b6df2ed [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 16:23:01.099061 11508 raft_consensus.cc:504] T d01444e43d0d4a639f0655acae1c1b44 P 57576785ec93441f9b827f3a8b6df2ed [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ba1dc75657f4418b6903fa6ceed3d8b" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 46397 } } peers { permanent_uuid: "274512f1889a4254b485fef854967321" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 42799 } } peers { permanent_uuid: "57576785ec93441f9b827f3a8b6df2ed" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 41875 } }
I20260809 16:23:01.099449 11508 leader_election.cc:288] T d01444e43d0d4a639f0655acae1c1b44 P 57576785ec93441f9b827f3a8b6df2ed [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5ba1dc75657f4418b6903fa6ceed3d8b (127.21.51.130:46397), 274512f1889a4254b485fef854967321 (127.21.51.131:42799)
I20260809 16:23:01.102869 11496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:23:01.099752 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.51.129:43863 (local address 127.21.51.131:42799)
0809 16:23:01.099799 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0809 16:23:01.099801 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:23:01.099812 (+ 11us) server_negotiation.cc:407] Connection header received
0809 16:23:01.099843 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:23:01.099845 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:23:01.099854 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:23:01.099923 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:23:01.100280 (+ 357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:23:01.101092 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:23:01.102070 (+ 978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:23:01.102461 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:23:01.102551 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:23:01.102560 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0809 16:23:01.102769 (+ 209us) server_negotiation.cc:299] Negotiation successful
0809 16:23:01.102810 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260809 16:23:01.102856 11067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:23:01.099651 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.131:42799 (local address 127.21.51.129:43863)
0809 16:23:01.099708 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0809 16:23:01.099719 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 16:23:01.099758 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:23:01.099935 (+ 177us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:23:01.099936 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:23:01.099938 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:23:01.100149 (+ 211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:23:01.100154 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:23:01.101229 (+ 1075us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:23:01.101230 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:23:01.101967 (+ 737us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:23:01.101969 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:23:01.102547 (+ 578us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:23:01.102548 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:23:01.102645 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:23:01.102648 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 16:23:01.102735 (+ 87us) client_negotiation.cc:240] Negotiation successful
0809 16:23:01.102781 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39}
I20260809 16:23:01.102985 11499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:23:01.100195 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.51.129:35089 (local address 127.21.51.130:46397)
0809 16:23:01.100281 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0809 16:23:01.100283 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:23:01.100291 (+ 8us) server_negotiation.cc:407] Connection header received
0809 16:23:01.100317 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:23:01.100318 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:23:01.100327 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:23:01.100384 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:23:01.100548 (+ 164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:23:01.101300 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:23:01.102222 (+ 922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:23:01.102620 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:23:01.102721 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:23:01.102730 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0809 16:23:01.102883 (+ 153us) server_negotiation.cc:299] Negotiation successful
0809 16:23:01.102924 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260809 16:23:01.102994 11514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:23:01.099655 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.130:46397 (local address 127.21.51.129:35089)
0809 16:23:01.100042 (+ 387us) negotiation.cc:107] Waiting for socket to connect
0809 16:23:01.100055 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 16:23:01.100117 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:23:01.100407 (+ 290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:23:01.100407 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:23:01.100409 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:23:01.100471 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:23:01.100472 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:23:01.101384 (+ 912us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:23:01.101385 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:23:01.102126 (+ 741us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:23:01.102128 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:23:01.102721 (+ 593us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:23:01.102722 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:23:01.102797 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:23:01.102801 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 16:23:01.102876 (+ 75us) client_negotiation.cc:240] Negotiation successful
0809 16:23:01.102913 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":370,"mutex_wait_us":14,"thread_start_us":168,"threads_started":1}
I20260809 16:23:01.103243 11411 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d01444e43d0d4a639f0655acae1c1b44" candidate_uuid: "57576785ec93441f9b827f3a8b6df2ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "274512f1889a4254b485fef854967321" is_pre_election: true
I20260809 16:23:01.103433 11411 raft_consensus.cc:2454] T d01444e43d0d4a639f0655acae1c1b44 P 274512f1889a4254b485fef854967321 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57576785ec93441f9b827f3a8b6df2ed in term 0.
I20260809 16:23:01.103396 11190 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d01444e43d0d4a639f0655acae1c1b44" candidate_uuid: "57576785ec93441f9b827f3a8b6df2ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ba1dc75657f4418b6903fa6ceed3d8b" is_pre_election: true
I20260809 16:23:01.103566 11190 raft_consensus.cc:2454] T d01444e43d0d4a639f0655acae1c1b44 P 5ba1dc75657f4418b6903fa6ceed3d8b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57576785ec93441f9b827f3a8b6df2ed in term 0.
I20260809 16:23:01.103700 10893 leader_election.cc:302] T d01444e43d0d4a639f0655acae1c1b44 P 57576785ec93441f9b827f3a8b6df2ed [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 274512f1889a4254b485fef854967321, 57576785ec93441f9b827f3a8b6df2ed; no voters:
W20260809 16:23:01.103858 11066 tablet.cc:2369] T d01444e43d0d4a639f0655acae1c1b44 P 57576785ec93441f9b827f3a8b6df2ed: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 16:23:01.103883 11508 raft_consensus.cc:2781] T d01444e43d0d4a639f0655acae1c1b44 P 57576785ec93441f9b827f3a8b6df2ed [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 16:23:01.103969 11508 raft_consensus.cc:482] T d01444e43d0d4a639f0655acae1c1b44 P 57576785ec93441f9b827f3a8b6df2ed [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 16:23:01.103999 11508 raft_consensus.cc:3037] T d01444e43d0d4a639f0655acae1c1b44 P 57576785ec93441f9b827f3a8b6df2ed [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:23:01.104766 11508 raft_consensus.cc:504] T d01444e43d0d4a639f0655acae1c1b44 P 57576785ec93441f9b827f3a8b6df2ed [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ba1dc75657f4418b6903fa6ceed3d8b" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 46397 } } peers { permanent_uuid: "274512f1889a4254b485fef854967321" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 42799 } } peers { permanent_uuid: "57576785ec93441f9b827f3a8b6df2ed" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 41875 } }
I20260809 16:23:01.104923 11508 leader_election.cc:288] T d01444e43d0d4a639f0655acae1c1b44 P 57576785ec93441f9b827f3a8b6df2ed [CANDIDATE]: Term 1 election: Requested vote from peers 5ba1dc75657f4418b6903fa6ceed3d8b (127.21.51.130:46397), 274512f1889a4254b485fef854967321 (127.21.51.131:42799)
I20260809 16:23:01.105098 11190 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d01444e43d0d4a639f0655acae1c1b44" candidate_uuid: "57576785ec93441f9b827f3a8b6df2ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ba1dc75657f4418b6903fa6ceed3d8b"
I20260809 16:23:01.105124 11411 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d01444e43d0d4a639f0655acae1c1b44" candidate_uuid: "57576785ec93441f9b827f3a8b6df2ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "274512f1889a4254b485fef854967321"
I20260809 16:23:01.105203 11190 raft_consensus.cc:3037] T d01444e43d0d4a639f0655acae1c1b44 P 5ba1dc75657f4418b6903fa6ceed3d8b [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:23:01.105201 11411 raft_consensus.cc:3037] T d01444e43d0d4a639f0655acae1c1b44 P 274512f1889a4254b485fef854967321 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:23:01.105923 11411 raft_consensus.cc:2454] T d01444e43d0d4a639f0655acae1c1b44 P 274512f1889a4254b485fef854967321 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57576785ec93441f9b827f3a8b6df2ed in term 1.
I20260809 16:23:01.105950 11190 raft_consensus.cc:2454] T d01444e43d0d4a639f0655acae1c1b44 P 5ba1dc75657f4418b6903fa6ceed3d8b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57576785ec93441f9b827f3a8b6df2ed in term 1.
I20260809 16:23:01.106099 10893 leader_election.cc:302] T d01444e43d0d4a639f0655acae1c1b44 P 57576785ec93441f9b827f3a8b6df2ed [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 274512f1889a4254b485fef854967321, 57576785ec93441f9b827f3a8b6df2ed; no voters:
I20260809 16:23:01.106180 11508 raft_consensus.cc:2781] T d01444e43d0d4a639f0655acae1c1b44 P 57576785ec93441f9b827f3a8b6df2ed [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:23:01.106356 11508 raft_consensus.cc:686] T d01444e43d0d4a639f0655acae1c1b44 P 57576785ec93441f9b827f3a8b6df2ed [term 1 LEADER]: Becoming Leader. State: Replica: 57576785ec93441f9b827f3a8b6df2ed, State: Running, Role: LEADER
I20260809 16:23:01.106462 11508 consensus_queue.cc:237] T d01444e43d0d4a639f0655acae1c1b44 P 57576785ec93441f9b827f3a8b6df2ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ba1dc75657f4418b6903fa6ceed3d8b" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 46397 } } peers { permanent_uuid: "274512f1889a4254b485fef854967321" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 42799 } } peers { permanent_uuid: "57576785ec93441f9b827f3a8b6df2ed" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 41875 } }
I20260809 16:23:01.107307 10777 catalog_manager.cc:5495] T d01444e43d0d4a639f0655acae1c1b44 P 57576785ec93441f9b827f3a8b6df2ed reported cstate change: term changed from 0 to 1, leader changed from <none> to 57576785ec93441f9b827f3a8b6df2ed (127.21.51.129). New cstate: current_term: 1 leader_uuid: "57576785ec93441f9b827f3a8b6df2ed" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ba1dc75657f4418b6903fa6ceed3d8b" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 46397 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "274512f1889a4254b485fef854967321" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 42799 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "57576785ec93441f9b827f3a8b6df2ed" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 41875 } health_report { overall_health: HEALTHY } } }
I20260809 16:23:01.117838 11500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:23:01.115291 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49792 (local address 127.21.51.129:41875)
0809 16:23:01.115346 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0809 16:23:01.115348 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:23:01.115359 (+ 11us) server_negotiation.cc:407] Connection header received
0809 16:23:01.115430 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:23:01.115431 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:23:01.115449 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:23:01.115519 (+ 70us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 16:23:01.115676 (+ 157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:23:01.116140 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:23:01.117075 (+ 935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:23:01.117259 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:23:01.117331 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:23:01.117500 (+ 169us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 16:23:01.117591 (+ 91us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 16:23:01.117654 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0809 16:23:01.117745 (+ 91us) server_negotiation.cc:299] Negotiation successful
0809 16:23:01.117763 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260809 16:23:01.119565 11190 raft_consensus.cc:1261] T d01444e43d0d4a639f0655acae1c1b44 P 5ba1dc75657f4418b6903fa6ceed3d8b [term 1 FOLLOWER]: Refusing update from remote peer 57576785ec93441f9b827f3a8b6df2ed: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 16:23:01.119568 11411 raft_consensus.cc:1261] T d01444e43d0d4a639f0655acae1c1b44 P 274512f1889a4254b485fef854967321 [term 1 FOLLOWER]: Refusing update from remote peer 57576785ec93441f9b827f3a8b6df2ed: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 16:23:01.119905 11508 consensus_queue.cc:1035] T d01444e43d0d4a639f0655acae1c1b44 P 57576785ec93441f9b827f3a8b6df2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "5ba1dc75657f4418b6903fa6ceed3d8b" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 46397 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 16:23:01.120064 11517 consensus_queue.cc:1035] T d01444e43d0d4a639f0655acae1c1b44 P 57576785ec93441f9b827f3a8b6df2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "274512f1889a4254b485fef854967321" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 42799 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 16:23:01.122299 11522 mvcc.cc:205] Tried to move back new op lower bound from 7316654412263133184 to 7316654412212756480. Current Snapshot: MvccSnapshot[applied={T|T < 7316654412263133184}]
I20260809 16:23:01.122952 11523 mvcc.cc:205] Tried to move back new op lower bound from 7316654412263133184 to 7316654412212756480. Current Snapshot: MvccSnapshot[applied={T|T < 7316654412263133184 or (T in {7316654412263133184})}]
I20260809 16:23:01.124667 10777 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55900:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 16:23:01.124807 10777 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55900:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 16:23:01.125571 10777 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3b5cfde595b240be93937bb3ae6a7190: Sending DeleteTablet for 3 replicas of tablet d01444e43d0d4a639f0655acae1c1b44
I20260809 16:23:01.125906 11158 tablet_service.cc:1514] Processing DeleteTablet for tablet d01444e43d0d4a639f0655acae1c1b44 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:23:01 UTC) from {username='kudu', principal='kudu/127.21.51.190@KRBTEST.COM'} at 127.0.0.1:60972
I20260809 16:23:01.126096 11374 tablet_service.cc:1514] Processing DeleteTablet for tablet d01444e43d0d4a639f0655acae1c1b44 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:23:01 UTC) from {username='kudu', principal='kudu/127.21.51.190@KRBTEST.COM'} at 127.0.0.1:57110
I20260809 16:23:01.126138 11534 tablet_replica.cc:330] T d01444e43d0d4a639f0655acae1c1b44 P 5ba1dc75657f4418b6903fa6ceed3d8b: stopping tablet replica
I20260809 16:23:01.126294 11534 raft_consensus.cc:2229] T d01444e43d0d4a639f0655acae1c1b44 P 5ba1dc75657f4418b6903fa6ceed3d8b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 16:23:01.126322 10965 tablet_service.cc:1514] Processing DeleteTablet for tablet d01444e43d0d4a639f0655acae1c1b44 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:23:01 UTC) from {username='kudu', principal='kudu/127.21.51.190@KRBTEST.COM'} at 127.0.0.1:49782
I20260809 16:23:01.126467 11534 raft_consensus.cc:2258] T d01444e43d0d4a639f0655acae1c1b44 P 5ba1dc75657f4418b6903fa6ceed3d8b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 16:23:01.126554 11537 tablet_replica.cc:330] T d01444e43d0d4a639f0655acae1c1b44 P 57576785ec93441f9b827f3a8b6df2ed: stopping tablet replica
I20260809 16:23:01.126600 21710 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrNqUCn/build/release/bin/kudu with pid 10871
I20260809 16:23:01.126654 11537 raft_consensus.cc:2229] T d01444e43d0d4a639f0655acae1c1b44 P 57576785ec93441f9b827f3a8b6df2ed [term 1 LEADER]: Raft consensus shutting down.
I20260809 16:23:01.133713 11534 ts_tablet_manager.cc:1905] T d01444e43d0d4a639f0655acae1c1b44 P 5ba1dc75657f4418b6903fa6ceed3d8b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 16:23:01.135648 11534 ts_tablet_manager.cc:1918] T d01444e43d0d4a639f0655acae1c1b44 P 5ba1dc75657f4418b6903fa6ceed3d8b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 16:23:01.135742 11534 log.cc:1196] T d01444e43d0d4a639f0655acae1c1b44 P 5ba1dc75657f4418b6903fa6ceed3d8b: Deleting WAL directory at /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292417722232-21710-0/minicluster-data/ts-1/wal/wals/d01444e43d0d4a639f0655acae1c1b44
I20260809 16:23:01.136018 11534 ts_tablet_manager.cc:1939] T d01444e43d0d4a639f0655acae1c1b44 P 5ba1dc75657f4418b6903fa6ceed3d8b: Deleting consensus metadata
I20260809 16:23:01.136451 10760 catalog_manager.cc:4841] TS 5ba1dc75657f4418b6903fa6ceed3d8b (127.21.51.130:46397): tablet d01444e43d0d4a639f0655acae1c1b44 (table test-table [id=6eed8c27546f492ea3609280d409f421]) successfully deleted
I20260809 16:23:01.137163 11536 tablet_replica.cc:330] T d01444e43d0d4a639f0655acae1c1b44 P 274512f1889a4254b485fef854967321: stopping tablet replica
I20260809 16:23:01.137286 11536 raft_consensus.cc:2229] T d01444e43d0d4a639f0655acae1c1b44 P 274512f1889a4254b485fef854967321 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 16:23:01.137403 11536 raft_consensus.cc:2258] T d01444e43d0d4a639f0655acae1c1b44 P 274512f1889a4254b485fef854967321 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 16:23:01.138656 11536 ts_tablet_manager.cc:1905] T d01444e43d0d4a639f0655acae1c1b44 P 274512f1889a4254b485fef854967321: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 16:23:01.140604 11536 ts_tablet_manager.cc:1918] T d01444e43d0d4a639f0655acae1c1b44 P 274512f1889a4254b485fef854967321: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 16:23:01.140712 11536 log.cc:1196] T d01444e43d0d4a639f0655acae1c1b44 P 274512f1889a4254b485fef854967321: Deleting WAL directory at /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292417722232-21710-0/minicluster-data/ts-2/wal/wals/d01444e43d0d4a639f0655acae1c1b44
I20260809 16:23:01.141001 11536 ts_tablet_manager.cc:1939] T d01444e43d0d4a639f0655acae1c1b44 P 274512f1889a4254b485fef854967321: Deleting consensus metadata
I20260809 16:23:01.141474 10754 catalog_manager.cc:4841] TS 274512f1889a4254b485fef854967321 (127.21.51.131:42799): tablet d01444e43d0d4a639f0655acae1c1b44 (table test-table [id=6eed8c27546f492ea3609280d409f421]) successfully deleted
W20260809 16:23:01.143327 10754 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.21.51.129:41875 (error 108)
I20260809 16:23:01.143638 21710 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrNqUCn/build/release/bin/kudu with pid 11072
I20260809 16:23:01.145331 11490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:23:01.143759 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.129:41875 (local address 127.0.0.1:49802)
0809 16:23:01.145248 (+ 1489us) negotiation.cc:107] Waiting for socket to connect
0809 16:23:01.145292 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.51.129:41875: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1458}
W20260809 16:23:01.146881 10754 catalog_manager.cc:4584] TS 57576785ec93441f9b827f3a8b6df2ed (127.21.51.129:41875): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet d01444e43d0d4a639f0655acae1c1b44: Network error: Client connection negotiation failed: client connection to 127.21.51.129:41875: connect: Connection refused (error 111)
I20260809 16:23:01.152136 21710 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrNqUCn/build/release/bin/kudu with pid 11282
I20260809 16:23:01.161643 21710 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrNqUCn/build/release/bin/kudu with pid 10732
I20260809 16:23:01.171205 21710 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrNqUCn/build/release/bin/kudu
/tmp/dist-test-taskrNqUCn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292417722232-21710-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292417722232-21710-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292417722232-21710-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292417722232-21710-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.51.190:36707
--webserver_interface=127.21.51.190
--webserver_port=45677
--keytab_file=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.190.keytab
--principal=kudu/127.21.51.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.51.148:40989
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.51.190:36707
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292417722232-21710-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292417722232-21710-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292417722232-21710-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:23:01.257997 11541 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:23:01.258224 11541 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:23:01.258248 11541 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:23:01.259449 11541 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 16:23:01.259485 11541 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:23:01.259500 11541 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:23:01.259513 11541 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 16:23:01.259526 11541 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 16:23:01.260943 11541 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.51.148:40989
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292417722232-21710-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292417722232-21710-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.21.51.190:36707
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.190.keytab
--principal=kudu/127.21.51.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.51.190:36707
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292417722232-21710-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.51.190
--webserver_port=45677
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11541
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292417722232-21710-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:23:01.261147 11541 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:23:01.261322 11541 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:23:01.270898 11558 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:23:01.270902 11556 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:23:01.270953 11541 server_base.cc:1034] running on GCE node
W20260809 16:23:01.270898 11554 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:23:01.271477 11541 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:23:01.271798 11541 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:23:01.272972 11541 hybrid_clock.cc:648] HybridClock initialized: now 1786292581272960 us; error 46 us; skew 500 ppm
Aug 09 16:23:01 dist-test-slave-s11t krb5kdc[10712](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292581, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.51.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:23:01.275668 11541 init.cc:377] Logged in from keytab as kudu/127.21.51.190@KRBTEST.COM (short username kudu)
I20260809 16:23:01.276170 11541 webserver.cc:460] Webserver started at http://127.21.51.190:45677/ using document root <none> and password file <none>
I20260809 16:23:01.276419 11541 fs_manager.cc:359] Metadata directory not provided
I20260809 16:23:01.276515 11541 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:23:01.277959 11541 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260809 16:23:01.278620 11569 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:23:01.278793 11541 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 16:23:01.278908 11541 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292417722232-21710-0/minicluster-data/master-0/data,/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292417722232-21710-0/minicluster-data/master-0/wal
uuid: "3b5cfde595b240be93937bb3ae6a7190"
format_stamp: "Formatted at 2026-08-09 16:23:00 on dist-test-slave-s11t"
server_key: "426457e3792cbbc285f1a9de711bb57f"
server_key_iv: "a6f0c5a824caf21e65252a259d7b5892"
server_key_version: "encryptionkey@0"
I20260809 16:23:01.279313 11541 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292417722232-21710-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292417722232-21710-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292417722232-21710-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:23:01.298102 11541 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:23:01.300489 11541 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:23:01.300662 11541 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:23:01.305126 11541 rpc_server.cc:307] RPC server started. Bound to: 127.21.51.190:36707
I20260809 16:23:01.305181 11651 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.51.190:36707 every 8 connection(s)
I20260809 16:23:01.305614 11541 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292417722232-21710-0/minicluster-data/master-0/data/info.pb
I20260809 16:23:01.306613 21710 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrNqUCn/build/release/bin/kudu as pid 11541
I20260809 16:23:01.306921 21710 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrNqUCn/build/release/bin/kudu
/tmp/dist-test-taskrNqUCn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292417722232-21710-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292417722232-21710-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292417722232-21710-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292417722232-21710-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.51.129:41875
--local_ip_for_outbound_sockets=127.21.51.129
--tserver_master_addrs=127.21.51.190:36707
--webserver_port=43581
--webserver_interface=127.21.51.129
--keytab_file=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.129.keytab
--principal=kudu/127.21.51.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.51.148:40989
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292417722232-21710-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292417722232-21710-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292417722232-21710-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 16:23:01.307502 11654 sys_catalog.cc:263] Verifying existing consensus state
I20260809 16:23:01.308202 11654 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3b5cfde595b240be93937bb3ae6a7190: Bootstrap starting.
I20260809 16:23:01.316659 11654 log.cc:824] T 00000000000000000000000000000000 P 3b5cfde595b240be93937bb3ae6a7190: Log is configured to *not* fsync() on all Append() calls
I20260809 16:23:01.318748 11654 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3b5cfde595b240be93937bb3ae6a7190: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 16:23:01.319075 11654 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3b5cfde595b240be93937bb3ae6a7190: Bootstrap complete.
I20260809 16:23:01.321561 11654 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3b5cfde595b240be93937bb3ae6a7190 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b5cfde595b240be93937bb3ae6a7190" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 36707 } }
I20260809 16:23:01.321784 11654 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3b5cfde595b240be93937bb3ae6a7190 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b5cfde595b240be93937bb3ae6a7190, State: Initialized, Role: FOLLOWER
I20260809 16:23:01.321951 11654 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3b5cfde595b240be93937bb3ae6a7190 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b5cfde595b240be93937bb3ae6a7190" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 36707 } }
I20260809 16:23:01.322006 11654 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3b5cfde595b240be93937bb3ae6a7190 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 16:23:01.322072 11654 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3b5cfde595b240be93937bb3ae6a7190 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 16:23:01.322122 11654 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3b5cfde595b240be93937bb3ae6a7190 [term 1 FOLLOWER]: Advancing to term 2
I20260809 16:23:01.323128 11654 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3b5cfde595b240be93937bb3ae6a7190 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b5cfde595b240be93937bb3ae6a7190" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 36707 } }
I20260809 16:23:01.323213 11654 leader_election.cc:302] T 00000000000000000000000000000000 P 3b5cfde595b240be93937bb3ae6a7190 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3b5cfde595b240be93937bb3ae6a7190; no voters:
I20260809 16:23:01.323451 11654 leader_election.cc:288] T 00000000000000000000000000000000 P 3b5cfde595b240be93937bb3ae6a7190 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260809 16:23:01.323494 11660 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3b5cfde595b240be93937bb3ae6a7190 [term 2 FOLLOWER]: Leader election won for term 2
I20260809 16:23:01.323608 11660 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3b5cfde595b240be93937bb3ae6a7190 [term 2 LEADER]: Becoming Leader. State: Replica: 3b5cfde595b240be93937bb3ae6a7190, State: Running, Role: LEADER
I20260809 16:23:01.323719 11654 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3b5cfde595b240be93937bb3ae6a7190 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 16:23:01.323709 11660 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3b5cfde595b240be93937bb3ae6a7190 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b5cfde595b240be93937bb3ae6a7190" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 36707 } }
I20260809 16:23:01.324158 11660 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3b5cfde595b240be93937bb3ae6a7190 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3b5cfde595b240be93937bb3ae6a7190. Latest consensus state: current_term: 2 leader_uuid: "3b5cfde595b240be93937bb3ae6a7190" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b5cfde595b240be93937bb3ae6a7190" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 36707 } } }
I20260809 16:23:01.324254 11660 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3b5cfde595b240be93937bb3ae6a7190 [sys.catalog]: This master's current role is: LEADER
I20260809 16:23:01.324604 11666 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 16:23:01.324156 11662 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3b5cfde595b240be93937bb3ae6a7190 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "3b5cfde595b240be93937bb3ae6a7190" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b5cfde595b240be93937bb3ae6a7190" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 36707 } } }
I20260809 16:23:01.324820 11662 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3b5cfde595b240be93937bb3ae6a7190 [sys.catalog]: This master's current role is: LEADER
I20260809 16:23:01.325498 11666 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 16:23:01.325636 11666 catalog_manager.cc:1259] Loaded cluster ID: 0c67fa9120874d83b449e4f732e4ad1a
I20260809 16:23:01.325675 11666 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 16:23:01.325956 11666 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260809 16:23:01.328869 11666 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260809 16:23:01.330629 11666 catalog_manager.cc:1504] Loading token signing keys...
I20260809 16:23:01.330845 11666 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 3b5cfde595b240be93937bb3ae6a7190: Loaded TSK: 0
I20260809 16:23:01.331071 11666 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260809 16:23:01.398350 11657 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:23:01.398558 11657 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:23:01.398581 11657 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:23:01.399758 11657 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:23:01.399802 11657 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:23:01.399828 11657 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.51.129
I20260809 16:23:01.401335 11657 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.51.148:40989
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292417722232-21710-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292417722232-21710-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.129.keytab
--principal=kudu/127.21.51.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.51.129:41875
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292417722232-21710-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.51.129
--webserver_port=43581
--tserver_master_addrs=127.21.51.190:36707
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11657
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.51.129
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292417722232-21710-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:23:01.401566 11657 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:23:01.401808 11657 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:23:01.406936 11699 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:23:01.406982 11704 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:23:01.407215 11657 server_base.cc:1034] running on GCE node
W20260809 16:23:01.407289 11706 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:23:01.407528 11657 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:23:01.407790 11657 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:23:01.408952 11657 hybrid_clock.cc:648] HybridClock initialized: now 1786292581408940 us; error 30 us; skew 500 ppm
Aug 09 16:23:01 dist-test-slave-s11t krb5kdc[10712](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292581, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.51.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:23:01.411501 11657 init.cc:377] Logged in from keytab as kudu/127.21.51.129@KRBTEST.COM (short username kudu)
I20260809 16:23:01.412078 11657 webserver.cc:460] Webserver started at http://127.21.51.129:43581/ using document root <none> and password file <none>
I20260809 16:23:01.412317 11657 fs_manager.cc:359] Metadata directory not provided
I20260809 16:23:01.412380 11657 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:23:01.413868 11657 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:23:01.414613 11715 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:23:01.414763 11657 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:23:01.414851 11657 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292417722232-21710-0/minicluster-data/ts-0/data,/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292417722232-21710-0/minicluster-data/ts-0/wal
uuid: "57576785ec93441f9b827f3a8b6df2ed"
format_stamp: "Formatted at 2026-08-09 16:23:00 on dist-test-slave-s11t"
server_key: "47e6daf2d4b2c8ffbb5ffec74976c636"
server_key_iv: "acafa1b1889fe54c341da0eec10112e2"
server_key_version: "encryptionkey@0"
I20260809 16:23:01.415161 11657 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292417722232-21710-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292417722232-21710-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292417722232-21710-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:23:01.441594 11657 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:23:01.444056 11657 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:23:01.444209 11657 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:23:01.444519 11657 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:23:01.445063 11725 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260809 16:23:01.446552 11657 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260809 16:23:01.446610 11657 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260809 16:23:01.446638 11657 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260809 16:23:01.447321 11657 ts_tablet_manager.cc:610] Registered 1 tablets
I20260809 16:23:01.447367 11657 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260809 16:23:01.447407 11725 tablet_bootstrap.cc:492] T d01444e43d0d4a639f0655acae1c1b44 P 57576785ec93441f9b827f3a8b6df2ed: Bootstrap starting.
I20260809 16:23:01.455155 11657 rpc_server.cc:307] RPC server started. Bound to: 127.21.51.129:41875
I20260809 16:23:01.455399 11876 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.51.129:41875 every 8 connection(s)
I20260809 16:23:01.455724 11657 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292417722232-21710-0/minicluster-data/ts-0/data/info.pb
I20260809 16:23:01.459224 11725 log.cc:824] T d01444e43d0d4a639f0655acae1c1b44 P 57576785ec93441f9b827f3a8b6df2ed: Log is configured to *not* fsync() on all Append() calls
I20260809 16:23:01.461578 11725 tablet_bootstrap.cc:492] T d01444e43d0d4a639f0655acae1c1b44 P 57576785ec93441f9b827f3a8b6df2ed: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 16:23:01.461897 11725 tablet_bootstrap.cc:492] T d01444e43d0d4a639f0655acae1c1b44 P 57576785ec93441f9b827f3a8b6df2ed: Bootstrap complete.
I20260809 16:23:01.463160 11725 ts_tablet_manager.cc:1397] T d01444e43d0d4a639f0655acae1c1b44 P 57576785ec93441f9b827f3a8b6df2ed: Time spent bootstrapping tablet: real 0.016s user 0.001s sys 0.011s
I20260809 16:23:01.463378 21710 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrNqUCn/build/release/bin/kudu as pid 11657
I20260809 16:23:01.463735 21710 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrNqUCn/build/release/bin/kudu
/tmp/dist-test-taskrNqUCn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292417722232-21710-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292417722232-21710-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292417722232-21710-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292417722232-21710-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.51.130:46397
--local_ip_for_outbound_sockets=127.21.51.130
--tserver_master_addrs=127.21.51.190:36707
--webserver_port=45661
--webserver_interface=127.21.51.130
--keytab_file=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.130.keytab
--principal=kudu/127.21.51.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.51.148:40989
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292417722232-21710-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292417722232-21710-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292417722232-21710-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 16:23:01.464491 11725 raft_consensus.cc:348] T d01444e43d0d4a639f0655acae1c1b44 P 57576785ec93441f9b827f3a8b6df2ed [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ba1dc75657f4418b6903fa6ceed3d8b" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 46397 } } peers { permanent_uuid: "274512f1889a4254b485fef854967321" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 42799 } } peers { permanent_uuid: "57576785ec93441f9b827f3a8b6df2ed" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 41875 } }
I20260809 16:23:01.464895 11725 raft_consensus.cc:729] T d01444e43d0d4a639f0655acae1c1b44 P 57576785ec93441f9b827f3a8b6df2ed [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57576785ec93441f9b827f3a8b6df2ed, State: Initialized, Role: FOLLOWER
I20260809 16:23:01.465070 11725 consensus_queue.cc:260] T d01444e43d0d4a639f0655acae1c1b44 P 57576785ec93441f9b827f3a8b6df2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ba1dc75657f4418b6903fa6ceed3d8b" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 46397 } } peers { permanent_uuid: "274512f1889a4254b485fef854967321" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 42799 } } peers { permanent_uuid: "57576785ec93441f9b827f3a8b6df2ed" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 41875 } }
I20260809 16:23:01.465358 11725 ts_tablet_manager.cc:1428] T d01444e43d0d4a639f0655acae1c1b44 P 57576785ec93441f9b827f3a8b6df2ed: Time spent starting tablet: real 0.002s user 0.005s sys 0.000s
Aug 09 16:23:01 dist-test-slave-s11t krb5kdc[10712](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292581, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.51.129@KRBTEST.COM for kudu/127.21.51.190@KRBTEST.COM
I20260809 16:23:01.470512 11882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:23:01.456639 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.51.129:41967 (local address 127.21.51.190:36707)
0809 16:23:01.459436 (+ 2797us) server_negotiation.cc:206] Beginning negotiation
0809 16:23:01.459445 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 16:23:01.459482 (+ 37us) server_negotiation.cc:407] Connection header received
0809 16:23:01.459611 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:23:01.459617 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:23:01.459669 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:23:01.459810 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:23:01.460662 (+ 852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:23:01.462020 (+ 1358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:23:01.463611 (+ 1591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:23:01.464223 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:23:01.464340 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:23:01.466762 (+ 2422us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:23:01.466766 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:23:01.466766 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:23:01.466778 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:23:01.468625 (+ 1847us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:23:01.469191 (+ 566us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:23:01.469194 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:23:01.469194 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:23:01.469298 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:23:01.469615 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:23:01.469616 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:23:01.469616 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:23:01.469832 (+ 216us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:23:01.469895 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0809 16:23:01.470270 (+ 375us) server_negotiation.cc:299] Negotiation successful
0809 16:23:01.470374 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2746,"thread_start_us":113,"threads_started":1}
I20260809 16:23:01.470662 11880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:23:01.456557 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:36707 (local address 127.21.51.129:41967)
0809 16:23:01.456777 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0809 16:23:01.456822 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0809 16:23:01.457597 (+ 775us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:23:01.459862 (+ 2265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:23:01.459866 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:23:01.460241 (+ 375us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:23:01.460517 (+ 276us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:23:01.460525 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:23:01.462143 (+ 1618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:23:01.462144 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:23:01.463510 (+ 1366us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:23:01.463513 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:23:01.464326 (+ 813us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:23:01.464327 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:23:01.464475 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:23:01.465100 (+ 625us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:23:01.465109 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:23:01.466638 (+ 1529us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:23:01.468746 (+ 2108us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:23:01.468754 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:23:01.468756 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:23:01.469099 (+ 343us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:23:01.469382 (+ 283us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:23:01.469383 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:23:01.469384 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:23:01.469538 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:23:01.469905 (+ 367us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:23:01.469912 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:23:01.470084 (+ 172us) client_negotiation.cc:769] Sending connection context
0809 16:23:01.470204 (+ 120us) client_negotiation.cc:240] Negotiation successful
0809 16:23:01.470410 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":122,"threads_started":1}
I20260809 16:23:01.471427 11877 heartbeater.cc:347] Connected to a master server at 127.21.51.190:36707
I20260809 16:23:01.471536 11877 heartbeater.cc:464] Registering TS with master...
I20260809 16:23:01.471752 11877 heartbeater.cc:511] Master 127.21.51.190:36707 requested a full tablet report, sending...
I20260809 16:23:01.472518 11591 ts_manager.cc:194] Registered new tserver with Master: 57576785ec93441f9b827f3a8b6df2ed (127.21.51.129:41875)
I20260809 16:23:01.473608 11591 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.51.129@KRBTEST.COM'} at 127.21.51.129:41967
I20260809 16:23:01.474419 11877 heartbeater.cc:503] Master 127.21.51.190:36707 was elected leader, sending a full tablet report...
Aug 09 16:23:01 dist-test-slave-s11t krb5kdc[10712](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292581, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.51.190@KRBTEST.COM for kudu/127.21.51.129@KRBTEST.COM
I20260809 16:23:01.482153 11888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:23:01.473238 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49808 (local address 127.21.51.129:41875)
0809 16:23:01.473388 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0809 16:23:01.473392 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 16:23:01.473580 (+ 188us) server_negotiation.cc:407] Connection header received
0809 16:23:01.473664 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:23:01.473667 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:23:01.473684 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:23:01.473752 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:23:01.474399 (+ 647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:23:01.475577 (+ 1178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:23:01.476836 (+ 1259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:23:01.477111 (+ 275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:23:01.477169 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:23:01.479241 (+ 2072us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:23:01.479248 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:23:01.479249 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:23:01.479256 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:23:01.480838 (+ 1582us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:23:01.481244 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:23:01.481247 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:23:01.481247 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:23:01.481309 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:23:01.481522 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:23:01.481523 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:23:01.481523 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:23:01.481661 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:23:01.481751 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0809 16:23:01.481976 (+ 225us) server_negotiation.cc:299] Negotiation successful
0809 16:23:01.482062 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":82,"threads_started":1}
I20260809 16:23:01.482152 11887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:23:01.473168 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.129:41875 (local address 127.0.0.1:49808)
0809 16:23:01.473457 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0809 16:23:01.473492 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0809 16:23:01.473570 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:23:01.473827 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:23:01.473834 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:23:01.474208 (+ 374us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:23:01.474274 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:23:01.474277 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:23:01.475699 (+ 1422us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:23:01.475700 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:23:01.476754 (+ 1054us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:23:01.476757 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:23:01.477188 (+ 431us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:23:01.477189 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:23:01.477268 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:23:01.477702 (+ 434us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:23:01.477707 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:23:01.479149 (+ 1442us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:23:01.480936 (+ 1787us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:23:01.480942 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:23:01.480944 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:23:01.481144 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:23:01.481386 (+ 242us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:23:01.481387 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:23:01.481387 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:23:01.481451 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:23:01.481733 (+ 282us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:23:01.481739 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:23:01.481796 (+ 57us) client_negotiation.cc:769] Sending connection context
0809 16:23:01.481898 (+ 102us) client_negotiation.cc:240] Negotiation successful
0809 16:23:01.482054 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":258,"thread_start_us":89,"threads_started":1}
I20260809 16:23:01.482566 11783 tablet_service.cc:1514] Processing DeleteTablet for tablet d01444e43d0d4a639f0655acae1c1b44 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:23:01 UTC) from {username='kudu', principal='kudu/127.21.51.190@KRBTEST.COM'} at 127.0.0.1:49808
I20260809 16:23:01.483073 11890 tablet_replica.cc:330] stopping tablet replica
I20260809 16:23:01.483182 11890 raft_consensus.cc:2229] T d01444e43d0d4a639f0655acae1c1b44 P 57576785ec93441f9b827f3a8b6df2ed [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 16:23:01.483247 11890 raft_consensus.cc:2258] T d01444e43d0d4a639f0655acae1c1b44 P 57576785ec93441f9b827f3a8b6df2ed [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 16:23:01.483557 11890 ts_tablet_manager.cc:1905] T d01444e43d0d4a639f0655acae1c1b44 P 57576785ec93441f9b827f3a8b6df2ed: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 16:23:01.486353 11890 ts_tablet_manager.cc:1918] T d01444e43d0d4a639f0655acae1c1b44 P 57576785ec93441f9b827f3a8b6df2ed: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 16:23:01.486440 11890 log.cc:1196] T d01444e43d0d4a639f0655acae1c1b44 P 57576785ec93441f9b827f3a8b6df2ed: Deleting WAL directory at /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292417722232-21710-0/minicluster-data/ts-0/wal/wals/d01444e43d0d4a639f0655acae1c1b44
I20260809 16:23:01.486678 11890 ts_tablet_manager.cc:1939] T d01444e43d0d4a639f0655acae1c1b44 P 57576785ec93441f9b827f3a8b6df2ed: Deleting consensus metadata
I20260809 16:23:01.487181 11570 catalog_manager.cc:4841] TS 57576785ec93441f9b827f3a8b6df2ed (127.21.51.129:41875): tablet d01444e43d0d4a639f0655acae1c1b44 (table test-table [id=6eed8c27546f492ea3609280d409f421]) successfully deleted
W20260809 16:23:01.551359 11886 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:23:01.551625 11886 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:23:01.551652 11886 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:23:01.552841 11886 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:23:01.552888 11886 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:23:01.552915 11886 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.51.130
I20260809 16:23:01.554364 11886 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.51.148:40989
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292417722232-21710-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292417722232-21710-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.130.keytab
--principal=kudu/127.21.51.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.51.130:46397
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292417722232-21710-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.51.130
--webserver_port=45661
--tserver_master_addrs=127.21.51.190:36707
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11886
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.51.130
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292417722232-21710-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:23:01.554605 11886 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:23:01.554829 11886 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:23:01.560024 11896 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:23:01.560050 11898 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:23:01.560065 11886 server_base.cc:1034] running on GCE node
W20260809 16:23:01.560046 11901 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:23:01.560467 11886 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:23:01.560698 11886 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:23:01.561913 11886 hybrid_clock.cc:648] HybridClock initialized: now 1786292581561900 us; error 30 us; skew 500 ppm
Aug 09 16:23:01 dist-test-slave-s11t krb5kdc[10712](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292581, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.51.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:23:01.564750 11886 init.cc:377] Logged in from keytab as kudu/127.21.51.130@KRBTEST.COM (short username kudu)
I20260809 16:23:01.565161 11886 webserver.cc:460] Webserver started at http://127.21.51.130:45661/ using document root <none> and password file <none>
I20260809 16:23:01.565420 11886 fs_manager.cc:359] Metadata directory not provided
I20260809 16:23:01.565506 11886 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:23:01.566838 11886 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:23:01.567497 11912 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:23:01.567684 11886 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:23:01.567793 11886 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292417722232-21710-0/minicluster-data/ts-1/data,/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292417722232-21710-0/minicluster-data/ts-1/wal
uuid: "5ba1dc75657f4418b6903fa6ceed3d8b"
format_stamp: "Formatted at 2026-08-09 16:23:00 on dist-test-slave-s11t"
server_key: "b1aee06ad35f0c32fe8b3fc4df3cfac2"
server_key_iv: "42725d4f5393d5976a7fc58d89e13374"
server_key_version: "encryptionkey@0"
I20260809 16:23:01.568120 11886 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292417722232-21710-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292417722232-21710-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292417722232-21710-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:23:01.605330 11886 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:23:01.607475 11886 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:23:01.607647 11886 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:23:01.607904 11886 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:23:01.608232 11886 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:23:01.608296 11886 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:23:01.608338 11886 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:23:01.608382 11886 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:23:01.614517 11886 rpc_server.cc:307] RPC server started. Bound to: 127.21.51.130:46397
I20260809 16:23:01.614564 12073 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.51.130:46397 every 8 connection(s)
I20260809 16:23:01.614912 11886 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292417722232-21710-0/minicluster-data/ts-1/data/info.pb
I20260809 16:23:01.620458 21710 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrNqUCn/build/release/bin/kudu as pid 11886
I20260809 16:23:01.620786 21710 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrNqUCn/build/release/bin/kudu
/tmp/dist-test-taskrNqUCn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292417722232-21710-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292417722232-21710-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292417722232-21710-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292417722232-21710-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.51.131:42799
--local_ip_for_outbound_sockets=127.21.51.131
--tserver_master_addrs=127.21.51.190:36707
--webserver_port=38313
--webserver_interface=127.21.51.131
--keytab_file=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.131.keytab
--principal=kudu/127.21.51.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.51.148:40989
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292417722232-21710-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292417722232-21710-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292417722232-21710-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 16:23:01 dist-test-slave-s11t krb5kdc[10712](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292581, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.51.130@KRBTEST.COM for kudu/127.21.51.190@KRBTEST.COM
I20260809 16:23:01.625134 11882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:23:01.615648 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.51.130:33269 (local address 127.21.51.190:36707)
0809 16:23:01.615729 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0809 16:23:01.615731 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:23:01.615957 (+ 226us) server_negotiation.cc:407] Connection header received
0809 16:23:01.616436 (+ 479us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:23:01.616438 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:23:01.616451 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:23:01.616512 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:23:01.617647 (+ 1135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:23:01.618189 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:23:01.619266 (+ 1077us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:23:01.619478 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:23:01.619534 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:23:01.621951 (+ 2417us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:23:01.621953 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:23:01.621953 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:23:01.621961 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:23:01.623448 (+ 1487us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:23:01.623993 (+ 545us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:23:01.623994 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:23:01.623994 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:23:01.624074 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:23:01.624351 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:23:01.624353 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:23:01.624353 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:23:01.624484 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:23:01.624565 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0809 16:23:01.624933 (+ 368us) server_negotiation.cc:299] Negotiation successful
0809 16:23:01.625040 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260809 16:23:01.625313 12077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:23:01.615625 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:36707 (local address 127.21.51.130:33269)
0809 16:23:01.615819 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0809 16:23:01.615861 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0809 16:23:01.616348 (+ 487us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:23:01.616593 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:23:01.616596 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:23:01.617343 (+ 747us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:23:01.617566 (+ 223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:23:01.617571 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:23:01.618354 (+ 783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:23:01.618355 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:23:01.619183 (+ 828us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:23:01.619186 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:23:01.619555 (+ 369us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:23:01.619556 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:23:01.619656 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:23:01.620131 (+ 475us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:23:01.620138 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:23:01.621826 (+ 1688us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:23:01.623526 (+ 1700us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:23:01.623532 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:23:01.623534 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:23:01.623913 (+ 379us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:23:01.624163 (+ 250us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:23:01.624164 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:23:01.624164 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:23:01.624281 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:23:01.624554 (+ 273us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:23:01.624561 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:23:01.624752 (+ 191us) client_negotiation.cc:769] Sending connection context
0809 16:23:01.624883 (+ 131us) client_negotiation.cc:240] Negotiation successful
0809 16:23:01.625051 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":110,"threads_started":1}
I20260809 16:23:01.625806 12074 heartbeater.cc:347] Connected to a master server at 127.21.51.190:36707
I20260809 16:23:01.625905 12074 heartbeater.cc:464] Registering TS with master...
I20260809 16:23:01.626106 12074 heartbeater.cc:511] Master 127.21.51.190:36707 requested a full tablet report, sending...
I20260809 16:23:01.626535 11591 ts_manager.cc:194] Registered new tserver with Master: 5ba1dc75657f4418b6903fa6ceed3d8b (127.21.51.130:46397)
I20260809 16:23:01.627128 11591 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.51.130@KRBTEST.COM'} at 127.21.51.130:33269
W20260809 16:23:01.703370 12079 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:23:01.703567 12079 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:23:01.703589 12079 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:23:01.704640 12079 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:23:01.704672 12079 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:23:01.704717 12079 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.51.131
I20260809 16:23:01.706045 12079 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.51.148:40989
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292417722232-21710-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292417722232-21710-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.131.keytab
--principal=kudu/127.21.51.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.51.131:42799
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292417722232-21710-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.51.131
--webserver_port=38313
--tserver_master_addrs=127.21.51.190:36707
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12079
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.51.131
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292417722232-21710-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:23:01.706218 12079 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:23:01.706470 12079 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:23:01.711401 12087 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:23:01.711442 12091 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:23:01.711411 12084 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:23:01.711439 12079 server_base.cc:1034] running on GCE node
I20260809 16:23:01.711794 12079 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:23:01.711988 12079 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:23:01.713160 12079 hybrid_clock.cc:648] HybridClock initialized: now 1786292581713145 us; error 30 us; skew 500 ppm
Aug 09 16:23:01 dist-test-slave-s11t krb5kdc[10712](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292581, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.51.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:23:01.715802 12079 init.cc:377] Logged in from keytab as kudu/127.21.51.131@KRBTEST.COM (short username kudu)
I20260809 16:23:01.716248 12079 webserver.cc:460] Webserver started at http://127.21.51.131:38313/ using document root <none> and password file <none>
I20260809 16:23:01.716481 12079 fs_manager.cc:359] Metadata directory not provided
I20260809 16:23:01.716569 12079 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:23:01.718065 12079 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:23:01.718786 12103 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:23:01.718977 12079 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:23:01.719075 12079 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292417722232-21710-0/minicluster-data/ts-2/data,/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292417722232-21710-0/minicluster-data/ts-2/wal
uuid: "274512f1889a4254b485fef854967321"
format_stamp: "Formatted at 2026-08-09 16:23:00 on dist-test-slave-s11t"
server_key: "5c0923b2d7f6c75f47c908daa014db10"
server_key_iv: "9d15cee54955387cc133adf55e93d3fc"
server_key_version: "encryptionkey@0"
I20260809 16:23:01.719411 12079 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292417722232-21710-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292417722232-21710-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292417722232-21710-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:23:01.733776 12079 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:23:01.735704 12079 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:23:01.735873 12079 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:23:01.736102 12079 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:23:01.736414 12079 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:23:01.736449 12079 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:23:01.736470 12079 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:23:01.736518 12079 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:23:01.742789 12079 rpc_server.cc:307] RPC server started. Bound to: 127.21.51.131:42799
I20260809 16:23:01.742862 12265 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.51.131:42799 every 8 connection(s)
I20260809 16:23:01.743248 12079 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292417722232-21710-0/minicluster-data/ts-2/data/info.pb
I20260809 16:23:01.746927 21710 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrNqUCn/build/release/bin/kudu as pid 12079
Aug 09 16:23:01 dist-test-slave-s11t krb5kdc[10712](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292581, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.51.131@KRBTEST.COM for kudu/127.21.51.190@KRBTEST.COM
I20260809 16:23:01.754238 11882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:23:01.744285 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.51.131:39561 (local address 127.21.51.190:36707)
0809 16:23:01.744330 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0809 16:23:01.744332 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:23:01.744569 (+ 237us) server_negotiation.cc:407] Connection header received
0809 16:23:01.745346 (+ 777us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:23:01.745348 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:23:01.745362 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:23:01.745420 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:23:01.746370 (+ 950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:23:01.746899 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:23:01.748224 (+ 1325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:23:01.748446 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:23:01.748525 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:23:01.751066 (+ 2541us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:23:01.751068 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:23:01.751068 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:23:01.751076 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:23:01.752726 (+ 1650us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:23:01.753198 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:23:01.753199 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:23:01.753199 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:23:01.753250 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:23:01.753528 (+ 278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:23:01.753529 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:23:01.753529 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:23:01.753667 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:23:01.753745 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0809 16:23:01.754070 (+ 325us) server_negotiation.cc:299] Negotiation successful
0809 16:23:01.754158 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260809 16:23:01.754643 12268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:23:01.744282 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:36707 (local address 127.21.51.131:39561)
0809 16:23:01.744446 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0809 16:23:01.744481 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0809 16:23:01.745223 (+ 742us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:23:01.745490 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:23:01.745495 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:23:01.746072 (+ 577us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:23:01.746265 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:23:01.746269 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:23:01.747005 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:23:01.747006 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:23:01.748127 (+ 1121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:23:01.748129 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:23:01.748529 (+ 400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:23:01.748530 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:23:01.748665 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:23:01.749261 (+ 596us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:23:01.749270 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:23:01.750994 (+ 1724us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:23:01.752802 (+ 1808us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:23:01.752808 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:23:01.752810 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:23:01.753114 (+ 304us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:23:01.753313 (+ 199us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:23:01.753314 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:23:01.753314 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:23:01.753454 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:23:01.753740 (+ 286us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:23:01.753745 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:23:01.753915 (+ 170us) client_negotiation.cc:769] Sending connection context
0809 16:23:01.754026 (+ 111us) client_negotiation.cc:240] Negotiation successful
0809 16:23:01.754201 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":89,"threads_started":1}
I20260809 16:23:01.754802 12271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:23:01.747381 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55908 (local address 127.21.51.190:36707)
0809 16:23:01.747711 (+ 330us) server_negotiation.cc:206] Beginning negotiation
0809 16:23:01.747713 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:23:01.747720 (+ 7us) server_negotiation.cc:407] Connection header received
0809 16:23:01.747741 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:23:01.747741 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:23:01.747751 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:23:01.747812 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:23:01.748430 (+ 618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:23:01.749071 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:23:01.750183 (+ 1112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:23:01.750512 (+ 329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:23:01.750587 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:23:01.751792 (+ 1205us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:23:01.751793 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:23:01.751793 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:23:01.751800 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:23:01.753428 (+ 1628us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:23:01.753849 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:23:01.753850 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:23:01.753850 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:23:01.753912 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:23:01.754137 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:23:01.754138 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:23:01.754139 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:23:01.754268 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:23:01.754435 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0809 16:23:01.754610 (+ 175us) server_negotiation.cc:299] Negotiation successful
0809 16:23:01.754696 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":310,"thread_start_us":60,"threads_started":1}
I20260809 16:23:01.755131 12266 heartbeater.cc:347] Connected to a master server at 127.21.51.190:36707
I20260809 16:23:01.755224 12266 heartbeater.cc:464] Registering TS with master...
I20260809 16:23:01.755404 12266 heartbeater.cc:511] Master 127.21.51.190:36707 requested a full tablet report, sending...
I20260809 16:23:01.755810 11591 ts_manager.cc:194] Registered new tserver with Master: 274512f1889a4254b485fef854967321 (127.21.51.131:42799)
I20260809 16:23:01.756145 21710 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 16:23:01.756402 11591 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.51.131@KRBTEST.COM'} at 127.21.51.131:39561
I20260809 16:23:01.758802 12271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:23:01.756607 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55912 (local address 127.21.51.190:36707)
0809 16:23:01.756714 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0809 16:23:01.756716 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:23:01.756725 (+ 9us) server_negotiation.cc:407] Connection header received
0809 16:23:01.756756 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:23:01.756757 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:23:01.756793 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:23:01.756896 (+ 103us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 16:23:01.757036 (+ 140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:23:01.757462 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:23:01.758149 (+ 687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:23:01.758328 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:23:01.758380 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:23:01.758567 (+ 187us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 16:23:01.758649 (+ 82us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 16:23:01.758703 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0809 16:23:01.758743 (+ 40us) server_negotiation.cc:299] Negotiation successful
0809 16:23:01.758761 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260809 16:23:01.759063 11591 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:55912:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 16:23:01.760527 11591 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 16:23:01.762630 11783 tablet_service.cc:1467] Processing CreateTablet for tablet 8e7536df343f4e51853cff79a145f428 (DEFAULT_TABLE table=test-table [id=7e7f4c4e49004c6fb6040bad53853944]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:23:01.762864 11783 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8e7536df343f4e51853cff79a145f428. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:23:01.764328 11725 tablet_bootstrap.cc:492] T 8e7536df343f4e51853cff79a145f428 P 57576785ec93441f9b827f3a8b6df2ed: Bootstrap starting.
I20260809 16:23:01.764638 11725 tablet_bootstrap.cc:654] T 8e7536df343f4e51853cff79a145f428 P 57576785ec93441f9b827f3a8b6df2ed: Neither blocks nor log segments found. Creating new log.
I20260809 16:23:01.765283 11725 tablet_bootstrap.cc:492] T 8e7536df343f4e51853cff79a145f428 P 57576785ec93441f9b827f3a8b6df2ed: No bootstrap required, opened a new log
I20260809 16:23:01.765331 11725 ts_tablet_manager.cc:1397] T 8e7536df343f4e51853cff79a145f428 P 57576785ec93441f9b827f3a8b6df2ed: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20260809 16:23:01.765673 11725 raft_consensus.cc:348] T 8e7536df343f4e51853cff79a145f428 P 57576785ec93441f9b827f3a8b6df2ed [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "274512f1889a4254b485fef854967321" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 42799 } } peers { permanent_uuid: "5ba1dc75657f4418b6903fa6ceed3d8b" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 46397 } } peers { permanent_uuid: "57576785ec93441f9b827f3a8b6df2ed" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 41875 } }
I20260809 16:23:01.765753 11725 raft_consensus.cc:374] T 8e7536df343f4e51853cff79a145f428 P 57576785ec93441f9b827f3a8b6df2ed [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:23:01.765798 11725 raft_consensus.cc:729] T 8e7536df343f4e51853cff79a145f428 P 57576785ec93441f9b827f3a8b6df2ed [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57576785ec93441f9b827f3a8b6df2ed, State: Initialized, Role: FOLLOWER
I20260809 16:23:01.765851 11725 consensus_queue.cc:260] T 8e7536df343f4e51853cff79a145f428 P 57576785ec93441f9b827f3a8b6df2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "274512f1889a4254b485fef854967321" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 42799 } } peers { permanent_uuid: "5ba1dc75657f4418b6903fa6ceed3d8b" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 46397 } } peers { permanent_uuid: "57576785ec93441f9b827f3a8b6df2ed" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 41875 } }
I20260809 16:23:01.765976 11725 ts_tablet_manager.cc:1428] T 8e7536df343f4e51853cff79a145f428 P 57576785ec93441f9b827f3a8b6df2ed: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260809 16:23:01.766461 11887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:23:01.762320 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.131:42799 (local address 127.0.0.1:57118)
0809 16:23:01.762429 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0809 16:23:01.762442 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 16:23:01.762492 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:23:01.762814 (+ 322us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:23:01.762815 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:23:01.762817 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:23:01.762869 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:23:01.762870 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:23:01.764144 (+ 1274us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:23:01.764145 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:23:01.765254 (+ 1109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:23:01.765257 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:23:01.766196 (+ 939us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:23:01.766197 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:23:01.766263 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:23:01.766278 (+ 15us) client_negotiation.cc:769] Sending connection context
0809 16:23:01.766337 (+ 59us) client_negotiation.cc:240] Negotiation successful
0809 16:23:01.766396 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":90}
I20260809 16:23:01.766472 12274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:23:01.762459 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57118 (local address 127.21.51.131:42799)
0809 16:23:01.762623 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0809 16:23:01.762628 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 16:23:01.762637 (+ 9us) server_negotiation.cc:407] Connection header received
0809 16:23:01.762680 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:23:01.762682 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:23:01.762703 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:23:01.762804 (+ 101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:23:01.762942 (+ 138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:23:01.764036 (+ 1094us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:23:01.765386 (+ 1350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:23:01.766116 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:23:01.766182 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:23:01.766235 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0809 16:23:01.766357 (+ 122us) server_negotiation.cc:299] Negotiation successful
0809 16:23:01.766395 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":137,"thread_start_us":77,"threads_started":1}
I20260809 16:23:01.767289 12273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:23:01.762319 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.130:46397 (local address 127.0.0.1:60976)
0809 16:23:01.762498 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0809 16:23:01.762509 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 16:23:01.762557 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:23:01.762997 (+ 440us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:23:01.762998 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:23:01.763000 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:23:01.763037 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:23:01.763038 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:23:01.764785 (+ 1747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:23:01.764786 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:23:01.765874 (+ 1088us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:23:01.765877 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:23:01.767049 (+ 1172us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:23:01.767050 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:23:01.767115 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:23:01.767119 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 16:23:01.767185 (+ 66us) client_negotiation.cc:240] Negotiation successful
0809 16:23:01.767223 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":162,"mutex_wait_us":21,"thread_start_us":98,"threads_started":1}
I20260809 16:23:01.767169 12175 tablet_service.cc:1467] Processing CreateTablet for tablet 8e7536df343f4e51853cff79a145f428 (DEFAULT_TABLE table=test-table [id=7e7f4c4e49004c6fb6040bad53853944]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:23:01.767467 12275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:23:01.762590 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60976 (local address 127.21.51.130:46397)
0809 16:23:01.762812 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0809 16:23:01.762817 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 16:23:01.762828 (+ 11us) server_negotiation.cc:407] Connection header received
0809 16:23:01.762866 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:23:01.762868 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:23:01.762883 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:23:01.762996 (+ 113us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:23:01.763098 (+ 102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:23:01.764211 (+ 1113us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:23:01.766124 (+ 1913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:23:01.766965 (+ 841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:23:01.767022 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:23:01.767057 (+ 35us) server_negotiation.cc:1035] Waiting for connection context
0809 16:23:01.767194 (+ 137us) server_negotiation.cc:299] Negotiation successful
0809 16:23:01.767252 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":182,"thread_start_us":119,"threads_started":1}
I20260809 16:23:01.767537 12175 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8e7536df343f4e51853cff79a145f428. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:23:01.768131 11975 tablet_service.cc:1467] Processing CreateTablet for tablet 8e7536df343f4e51853cff79a145f428 (DEFAULT_TABLE table=test-table [id=7e7f4c4e49004c6fb6040bad53853944]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:23:01.768450 11975 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8e7536df343f4e51853cff79a145f428. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:23:01.771157 12279 tablet_bootstrap.cc:492] T 8e7536df343f4e51853cff79a145f428 P 274512f1889a4254b485fef854967321: Bootstrap starting.
I20260809 16:23:01.771768 12279 tablet_bootstrap.cc:654] T 8e7536df343f4e51853cff79a145f428 P 274512f1889a4254b485fef854967321: Neither blocks nor log segments found. Creating new log.
I20260809 16:23:01.771857 12280 tablet_bootstrap.cc:492] T 8e7536df343f4e51853cff79a145f428 P 5ba1dc75657f4418b6903fa6ceed3d8b: Bootstrap starting.
I20260809 16:23:01.772302 12279 log.cc:824] T 8e7536df343f4e51853cff79a145f428 P 274512f1889a4254b485fef854967321: Log is configured to *not* fsync() on all Append() calls
I20260809 16:23:01.772441 12280 tablet_bootstrap.cc:654] T 8e7536df343f4e51853cff79a145f428 P 5ba1dc75657f4418b6903fa6ceed3d8b: Neither blocks nor log segments found. Creating new log.
I20260809 16:23:01.773036 12280 log.cc:824] T 8e7536df343f4e51853cff79a145f428 P 5ba1dc75657f4418b6903fa6ceed3d8b: Log is configured to *not* fsync() on all Append() calls
I20260809 16:23:01.773238 12279 tablet_bootstrap.cc:492] T 8e7536df343f4e51853cff79a145f428 P 274512f1889a4254b485fef854967321: No bootstrap required, opened a new log
I20260809 16:23:01.773317 12279 ts_tablet_manager.cc:1397] T 8e7536df343f4e51853cff79a145f428 P 274512f1889a4254b485fef854967321: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 16:23:01.773712 12280 tablet_bootstrap.cc:492] T 8e7536df343f4e51853cff79a145f428 P 5ba1dc75657f4418b6903fa6ceed3d8b: No bootstrap required, opened a new log
I20260809 16:23:01.773764 12280 ts_tablet_manager.cc:1397] T 8e7536df343f4e51853cff79a145f428 P 5ba1dc75657f4418b6903fa6ceed3d8b: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260809 16:23:01.774883 12279 raft_consensus.cc:348] T 8e7536df343f4e51853cff79a145f428 P 274512f1889a4254b485fef854967321 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "274512f1889a4254b485fef854967321" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 42799 } } peers { permanent_uuid: "5ba1dc75657f4418b6903fa6ceed3d8b" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 46397 } } peers { permanent_uuid: "57576785ec93441f9b827f3a8b6df2ed" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 41875 } }
I20260809 16:23:01.775035 12279 raft_consensus.cc:374] T 8e7536df343f4e51853cff79a145f428 P 274512f1889a4254b485fef854967321 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:23:01.775081 12279 raft_consensus.cc:729] T 8e7536df343f4e51853cff79a145f428 P 274512f1889a4254b485fef854967321 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 274512f1889a4254b485fef854967321, State: Initialized, Role: FOLLOWER
I20260809 16:23:01.775197 12279 consensus_queue.cc:260] T 8e7536df343f4e51853cff79a145f428 P 274512f1889a4254b485fef854967321 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "274512f1889a4254b485fef854967321" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 42799 } } peers { permanent_uuid: "5ba1dc75657f4418b6903fa6ceed3d8b" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 46397 } } peers { permanent_uuid: "57576785ec93441f9b827f3a8b6df2ed" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 41875 } }
I20260809 16:23:01.775475 12279 ts_tablet_manager.cc:1428] T 8e7536df343f4e51853cff79a145f428 P 274512f1889a4254b485fef854967321: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260809 16:23:01.775506 12266 heartbeater.cc:503] Master 127.21.51.190:36707 was elected leader, sending a full tablet report...
I20260809 16:23:01.775668 12280 raft_consensus.cc:348] T 8e7536df343f4e51853cff79a145f428 P 5ba1dc75657f4418b6903fa6ceed3d8b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "274512f1889a4254b485fef854967321" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 42799 } } peers { permanent_uuid: "5ba1dc75657f4418b6903fa6ceed3d8b" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 46397 } } peers { permanent_uuid: "57576785ec93441f9b827f3a8b6df2ed" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 41875 } }
I20260809 16:23:01.775900 12280 raft_consensus.cc:374] T 8e7536df343f4e51853cff79a145f428 P 5ba1dc75657f4418b6903fa6ceed3d8b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:23:01.775949 12280 raft_consensus.cc:729] T 8e7536df343f4e51853cff79a145f428 P 5ba1dc75657f4418b6903fa6ceed3d8b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5ba1dc75657f4418b6903fa6ceed3d8b, State: Initialized, Role: FOLLOWER
I20260809 16:23:01.776060 12280 consensus_queue.cc:260] T 8e7536df343f4e51853cff79a145f428 P 5ba1dc75657f4418b6903fa6ceed3d8b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "274512f1889a4254b485fef854967321" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 42799 } } peers { permanent_uuid: "5ba1dc75657f4418b6903fa6ceed3d8b" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 46397 } } peers { permanent_uuid: "57576785ec93441f9b827f3a8b6df2ed" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 41875 } }
I20260809 16:23:01.776331 12280 ts_tablet_manager.cc:1428] T 8e7536df343f4e51853cff79a145f428 P 5ba1dc75657f4418b6903fa6ceed3d8b: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260809 16:23:01.776355 12074 heartbeater.cc:503] Master 127.21.51.190:36707 was elected leader, sending a full tablet report...
I20260809 16:23:01.813570 12283 raft_consensus.cc:482] T 8e7536df343f4e51853cff79a145f428 P 274512f1889a4254b485fef854967321 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 16:23:01.813696 12283 raft_consensus.cc:504] T 8e7536df343f4e51853cff79a145f428 P 274512f1889a4254b485fef854967321 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "274512f1889a4254b485fef854967321" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 42799 } } peers { permanent_uuid: "5ba1dc75657f4418b6903fa6ceed3d8b" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 46397 } } peers { permanent_uuid: "57576785ec93441f9b827f3a8b6df2ed" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 41875 } }
I20260809 16:23:01.814034 12283 leader_election.cc:288] T 8e7536df343f4e51853cff79a145f428 P 274512f1889a4254b485fef854967321 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5ba1dc75657f4418b6903fa6ceed3d8b (127.21.51.130:46397), 57576785ec93441f9b827f3a8b6df2ed (127.21.51.129:41875)
I20260809 16:23:01.817878 12268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:23:01.814231 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.130:46397 (local address 127.21.51.131:48187)
0809 16:23:01.814300 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0809 16:23:01.814320 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 16:23:01.814394 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:23:01.814567 (+ 173us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:23:01.814568 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:23:01.814570 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:23:01.814653 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:23:01.814656 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:23:01.815595 (+ 939us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:23:01.815596 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:23:01.816837 (+ 1241us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:23:01.816840 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:23:01.817592 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:23:01.817593 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:23:01.817658 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:23:01.817662 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 16:23:01.817738 (+ 76us) client_negotiation.cc:240] Negotiation successful
0809 16:23:01.817784 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":46,"mutex_wait_us":25}
I20260809 16:23:01.817910 12275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:23:01.814372 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.51.131:48187 (local address 127.21.51.130:46397)
0809 16:23:01.814427 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0809 16:23:01.814429 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:23:01.814440 (+ 11us) server_negotiation.cc:407] Connection header received
0809 16:23:01.814471 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:23:01.814472 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:23:01.814483 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:23:01.814547 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:23:01.814736 (+ 189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:23:01.815515 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:23:01.816954 (+ 1439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:23:01.817518 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:23:01.817595 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:23:01.817604 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0809 16:23:01.817765 (+ 161us) server_negotiation.cc:299] Negotiation successful
0809 16:23:01.817807 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260809 16:23:01.818008 11888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:23:01.814333 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.51.131:54979 (local address 127.21.51.129:41875)
0809 16:23:01.814400 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0809 16:23:01.814402 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:23:01.814671 (+ 269us) server_negotiation.cc:407] Connection header received
0809 16:23:01.814723 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:23:01.814724 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:23:01.814751 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:23:01.814823 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:23:01.814962 (+ 139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:23:01.815669 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:23:01.816970 (+ 1301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:23:01.817636 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:23:01.817694 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:23:01.817793 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0809 16:23:01.817883 (+ 90us) server_negotiation.cc:299] Negotiation successful
0809 16:23:01.817941 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260809 16:23:01.818243 12286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:23:01.814234 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.129:41875 (local address 127.21.51.131:54979)
0809 16:23:01.814587 (+ 353us) negotiation.cc:107] Waiting for socket to connect
0809 16:23:01.814597 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 16:23:01.814658 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:23:01.814834 (+ 176us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:23:01.814834 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:23:01.814836 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:23:01.814885 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:23:01.814886 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:23:01.815733 (+ 847us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:23:01.815734 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:23:01.816891 (+ 1157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:23:01.816894 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:23:01.817707 (+ 813us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:23:01.817708 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:23:01.817775 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:23:01.817779 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 16:23:01.818116 (+ 337us) client_negotiation.cc:240] Negotiation successful
0809 16:23:01.818158 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":334,"mutex_wait_us":161,"thread_start_us":106,"threads_started":1}
I20260809 16:23:01.818338 12008 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8e7536df343f4e51853cff79a145f428" candidate_uuid: "274512f1889a4254b485fef854967321" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ba1dc75657f4418b6903fa6ceed3d8b" is_pre_election: true
I20260809 16:23:01.818557 12008 raft_consensus.cc:2454] T 8e7536df343f4e51853cff79a145f428 P 5ba1dc75657f4418b6903fa6ceed3d8b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 274512f1889a4254b485fef854967321 in term 0.
I20260809 16:23:01.818562 11805 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8e7536df343f4e51853cff79a145f428" candidate_uuid: "274512f1889a4254b485fef854967321" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57576785ec93441f9b827f3a8b6df2ed" is_pre_election: true
I20260809 16:23:01.818743 11805 raft_consensus.cc:2454] T 8e7536df343f4e51853cff79a145f428 P 57576785ec93441f9b827f3a8b6df2ed [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 274512f1889a4254b485fef854967321 in term 0.
I20260809 16:23:01.818817 12109 leader_election.cc:302] T 8e7536df343f4e51853cff79a145f428 P 274512f1889a4254b485fef854967321 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 274512f1889a4254b485fef854967321, 5ba1dc75657f4418b6903fa6ceed3d8b; no voters:
I20260809 16:23:01.818931 12283 raft_consensus.cc:2781] T 8e7536df343f4e51853cff79a145f428 P 274512f1889a4254b485fef854967321 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 16:23:01.818986 12283 raft_consensus.cc:482] T 8e7536df343f4e51853cff79a145f428 P 274512f1889a4254b485fef854967321 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 16:23:01.819011 12283 raft_consensus.cc:3037] T 8e7536df343f4e51853cff79a145f428 P 274512f1889a4254b485fef854967321 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:23:01.819737 12283 raft_consensus.cc:504] T 8e7536df343f4e51853cff79a145f428 P 274512f1889a4254b485fef854967321 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "274512f1889a4254b485fef854967321" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 42799 } } peers { permanent_uuid: "5ba1dc75657f4418b6903fa6ceed3d8b" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 46397 } } peers { permanent_uuid: "57576785ec93441f9b827f3a8b6df2ed" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 41875 } }
I20260809 16:23:01.819881 12283 leader_election.cc:288] T 8e7536df343f4e51853cff79a145f428 P 274512f1889a4254b485fef854967321 [CANDIDATE]: Term 1 election: Requested vote from peers 5ba1dc75657f4418b6903fa6ceed3d8b (127.21.51.130:46397), 57576785ec93441f9b827f3a8b6df2ed (127.21.51.129:41875)
I20260809 16:23:01.820042 12008 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8e7536df343f4e51853cff79a145f428" candidate_uuid: "274512f1889a4254b485fef854967321" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ba1dc75657f4418b6903fa6ceed3d8b"
I20260809 16:23:01.820040 11805 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8e7536df343f4e51853cff79a145f428" candidate_uuid: "274512f1889a4254b485fef854967321" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57576785ec93441f9b827f3a8b6df2ed"
I20260809 16:23:01.820129 12008 raft_consensus.cc:3037] T 8e7536df343f4e51853cff79a145f428 P 5ba1dc75657f4418b6903fa6ceed3d8b [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:23:01.820139 11805 raft_consensus.cc:3037] T 8e7536df343f4e51853cff79a145f428 P 57576785ec93441f9b827f3a8b6df2ed [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:23:01.820955 11805 raft_consensus.cc:2454] T 8e7536df343f4e51853cff79a145f428 P 57576785ec93441f9b827f3a8b6df2ed [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 274512f1889a4254b485fef854967321 in term 1.
I20260809 16:23:01.820989 12008 raft_consensus.cc:2454] T 8e7536df343f4e51853cff79a145f428 P 5ba1dc75657f4418b6903fa6ceed3d8b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 274512f1889a4254b485fef854967321 in term 1.
I20260809 16:23:01.821152 12106 leader_election.cc:302] T 8e7536df343f4e51853cff79a145f428 P 274512f1889a4254b485fef854967321 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 274512f1889a4254b485fef854967321, 57576785ec93441f9b827f3a8b6df2ed; no voters:
I20260809 16:23:01.821260 12283 raft_consensus.cc:2781] T 8e7536df343f4e51853cff79a145f428 P 274512f1889a4254b485fef854967321 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:23:01.821434 12283 raft_consensus.cc:686] T 8e7536df343f4e51853cff79a145f428 P 274512f1889a4254b485fef854967321 [term 1 LEADER]: Becoming Leader. State: Replica: 274512f1889a4254b485fef854967321, State: Running, Role: LEADER
I20260809 16:23:01.821548 12283 consensus_queue.cc:237] T 8e7536df343f4e51853cff79a145f428 P 274512f1889a4254b485fef854967321 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "274512f1889a4254b485fef854967321" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 42799 } } peers { permanent_uuid: "5ba1dc75657f4418b6903fa6ceed3d8b" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 46397 } } peers { permanent_uuid: "57576785ec93441f9b827f3a8b6df2ed" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 41875 } }
I20260809 16:23:01.822467 11591 catalog_manager.cc:5495] T 8e7536df343f4e51853cff79a145f428 P 274512f1889a4254b485fef854967321 reported cstate change: term changed from 0 to 1, leader changed from <none> to 274512f1889a4254b485fef854967321 (127.21.51.131). New cstate: current_term: 1 leader_uuid: "274512f1889a4254b485fef854967321" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "274512f1889a4254b485fef854967321" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 42799 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5ba1dc75657f4418b6903fa6ceed3d8b" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 46397 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "57576785ec93441f9b827f3a8b6df2ed" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 41875 } health_report { overall_health: UNKNOWN } } }
I20260809 16:23:01.840827 12274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:23:01.837879 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57128 (local address 127.21.51.131:42799)
0809 16:23:01.837933 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0809 16:23:01.837935 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:23:01.837946 (+ 11us) server_negotiation.cc:407] Connection header received
0809 16:23:01.838029 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:23:01.838030 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:23:01.838051 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:23:01.838132 (+ 81us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 16:23:01.838301 (+ 169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:23:01.838926 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:23:01.840017 (+ 1091us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:23:01.840284 (+ 267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:23:01.840340 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:23:01.840481 (+ 141us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 16:23:01.840580 (+ 99us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 16:23:01.840637 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0809 16:23:01.840675 (+ 38us) server_negotiation.cc:299] Negotiation successful
0809 16:23:01.840737 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260809 16:23:01.842785 11805 raft_consensus.cc:1261] T 8e7536df343f4e51853cff79a145f428 P 57576785ec93441f9b827f3a8b6df2ed [term 1 FOLLOWER]: Refusing update from remote peer 274512f1889a4254b485fef854967321: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 16:23:01.843098 12008 raft_consensus.cc:1261] T 8e7536df343f4e51853cff79a145f428 P 5ba1dc75657f4418b6903fa6ceed3d8b [term 1 FOLLOWER]: Refusing update from remote peer 274512f1889a4254b485fef854967321: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 16:23:01.843132 12288 consensus_queue.cc:1035] T 8e7536df343f4e51853cff79a145f428 P 274512f1889a4254b485fef854967321 [LEADER]: Connected to new peer: Peer: permanent_uuid: "57576785ec93441f9b827f3a8b6df2ed" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 41875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 16:23:01.843360 12288 consensus_queue.cc:1035] T 8e7536df343f4e51853cff79a145f428 P 274512f1889a4254b485fef854967321 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5ba1dc75657f4418b6903fa6ceed3d8b" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 46397 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 16:23:01.848047 11591 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:55912:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 16:23:01.848150 11591 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:55912:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 16:23:01.848937 11591 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3b5cfde595b240be93937bb3ae6a7190: Sending DeleteTablet for 3 replicas of tablet 8e7536df343f4e51853cff79a145f428
I20260809 16:23:01.849261 11975 tablet_service.cc:1514] Processing DeleteTablet for tablet 8e7536df343f4e51853cff79a145f428 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:23:01 UTC) from {username='kudu', principal='kudu/127.21.51.190@KRBTEST.COM'} at 127.0.0.1:60976
I20260809 16:23:01.849308 11783 tablet_service.cc:1514] Processing DeleteTablet for tablet 8e7536df343f4e51853cff79a145f428 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:23:01 UTC) from {username='kudu', principal='kudu/127.21.51.190@KRBTEST.COM'} at 127.0.0.1:49808
I20260809 16:23:01.849313 12175 tablet_service.cc:1514] Processing DeleteTablet for tablet 8e7536df343f4e51853cff79a145f428 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:23:01 UTC) from {username='kudu', principal='kudu/127.21.51.190@KRBTEST.COM'} at 127.0.0.1:57118
I20260809 16:23:01.849473 11890 tablet_replica.cc:330] T 8e7536df343f4e51853cff79a145f428 P 57576785ec93441f9b827f3a8b6df2ed: stopping tablet replica
I20260809 16:23:01.849498 12307 tablet_replica.cc:330] T 8e7536df343f4e51853cff79a145f428 P 5ba1dc75657f4418b6903fa6ceed3d8b: stopping tablet replica
I20260809 16:23:01.849532 11890 raft_consensus.cc:2229] T 8e7536df343f4e51853cff79a145f428 P 57576785ec93441f9b827f3a8b6df2ed [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 16:23:01.849576 11890 raft_consensus.cc:2258] T 8e7536df343f4e51853cff79a145f428 P 57576785ec93441f9b827f3a8b6df2ed [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 16:23:01.849588 12307 raft_consensus.cc:2229] T 8e7536df343f4e51853cff79a145f428 P 5ba1dc75657f4418b6903fa6ceed3d8b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 16:23:01.849681 12307 raft_consensus.cc:2258] T 8e7536df343f4e51853cff79a145f428 P 5ba1dc75657f4418b6903fa6ceed3d8b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 16:23:01.849771 21710 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrNqUCn/build/release/bin/kudu with pid 11657
I20260809 16:23:01.849988 11890 ts_tablet_manager.cc:1905] T 8e7536df343f4e51853cff79a145f428 P 57576785ec93441f9b827f3a8b6df2ed: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 16:23:01.850138 12307 ts_tablet_manager.cc:1905] T 8e7536df343f4e51853cff79a145f428 P 5ba1dc75657f4418b6903fa6ceed3d8b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 16:23:01.853826 12307 ts_tablet_manager.cc:1918] T 8e7536df343f4e51853cff79a145f428 P 5ba1dc75657f4418b6903fa6ceed3d8b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 16:23:01.853901 12307 log.cc:1196] T 8e7536df343f4e51853cff79a145f428 P 5ba1dc75657f4418b6903fa6ceed3d8b: Deleting WAL directory at /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292417722232-21710-0/minicluster-data/ts-1/wal/wals/8e7536df343f4e51853cff79a145f428
I20260809 16:23:01.854194 12307 ts_tablet_manager.cc:1939] T 8e7536df343f4e51853cff79a145f428 P 5ba1dc75657f4418b6903fa6ceed3d8b: Deleting consensus metadata
I20260809 16:23:01.855381 11573 catalog_manager.cc:4841] TS 5ba1dc75657f4418b6903fa6ceed3d8b (127.21.51.130:46397): tablet 8e7536df343f4e51853cff79a145f428 (table test-table [id=7e7f4c4e49004c6fb6040bad53853944]) successfully deleted
I20260809 16:23:01.858014 12308 tablet_replica.cc:330] T 8e7536df343f4e51853cff79a145f428 P 274512f1889a4254b485fef854967321: stopping tablet replica
I20260809 16:23:01.858201 12308 raft_consensus.cc:2229] T 8e7536df343f4e51853cff79a145f428 P 274512f1889a4254b485fef854967321 [term 1 LEADER]: Raft consensus shutting down.
I20260809 16:23:01.858438 12308 raft_consensus.cc:2258] T 8e7536df343f4e51853cff79a145f428 P 274512f1889a4254b485fef854967321 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 16:23:01.859876 12308 ts_tablet_manager.cc:1905] T 8e7536df343f4e51853cff79a145f428 P 274512f1889a4254b485fef854967321: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 16:23:01.862197 12308 ts_tablet_manager.cc:1918] T 8e7536df343f4e51853cff79a145f428 P 274512f1889a4254b485fef854967321: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 16:23:01.862319 12308 log.cc:1196] T 8e7536df343f4e51853cff79a145f428 P 274512f1889a4254b485fef854967321: Deleting WAL directory at /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292417722232-21710-0/minicluster-data/ts-2/wal/wals/8e7536df343f4e51853cff79a145f428
I20260809 16:23:01.862601 12308 ts_tablet_manager.cc:1939] T 8e7536df343f4e51853cff79a145f428 P 274512f1889a4254b485fef854967321: Deleting consensus metadata
I20260809 16:23:01.863060 11570 catalog_manager.cc:4841] TS 274512f1889a4254b485fef854967321 (127.21.51.131:42799): tablet 8e7536df343f4e51853cff79a145f428 (table test-table [id=7e7f4c4e49004c6fb6040bad53853944]) successfully deleted
W20260809 16:23:01.866888 11570 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.21.51.129:41875 (error 108)
I20260809 16:23:01.867190 21710 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrNqUCn/build/release/bin/kudu with pid 11886
I20260809 16:23:01.872963 12273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:23:01.867317 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.129:41875 (local address 127.0.0.1:49814)
0809 16:23:01.872851 (+ 5534us) negotiation.cc:107] Waiting for socket to connect
0809 16:23:01.872914 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.51.129:41875: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":5504}
W20260809 16:23:01.885071 11570 catalog_manager.cc:4584] TS 57576785ec93441f9b827f3a8b6df2ed (127.21.51.129:41875): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 8e7536df343f4e51853cff79a145f428: Network error: Client connection negotiation failed: client connection to 127.21.51.129:41875: connect: Connection refused (error 111)
I20260809 16:23:01.886932 21710 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrNqUCn/build/release/bin/kudu with pid 12079
I20260809 16:23:01.900120 21710 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrNqUCn/build/release/bin/kudu with pid 11541
I20260809 16:23:01.910439 21710 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrNqUCn/build/release/bin/kudu
/tmp/dist-test-taskrNqUCn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292417722232-21710-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292417722232-21710-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292417722232-21710-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292417722232-21710-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.51.190:36707
--webserver_interface=127.21.51.190
--webserver_port=45677
--keytab_file=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.190.keytab
--principal=kudu/127.21.51.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.51.148:40989
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.51.190:36707
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292417722232-21710-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292417722232-21710-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292417722232-21710-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:23:01.993417 12312 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:23:01.993650 12312 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:23:01.993687 12312 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:23:01.994803 12312 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 16:23:01.994871 12312 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:23:01.994895 12312 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:23:01.994915 12312 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 16:23:01.994947 12312 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 16:23:01.996425 12312 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.51.148:40989
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292417722232-21710-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292417722232-21710-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.51.190:36707
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.190.keytab
--principal=kudu/127.21.51.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.51.190:36707
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292417722232-21710-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.51.190
--webserver_port=45677
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12312
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292417722232-21710-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:23:01.996722 12312 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:23:01.996968 12312 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:23:02.001973 12327 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:23:02.001983 12322 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:23:02.002069 12312 server_base.cc:1034] running on GCE node
W20260809 16:23:02.001979 12324 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:23:02.002492 12312 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:23:02.002842 12312 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:23:02.004016 12312 hybrid_clock.cc:648] HybridClock initialized: now 1786292582004002 us; error 30 us; skew 500 ppm
Aug 09 16:23:02 dist-test-slave-s11t krb5kdc[10712](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292582, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.51.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:23:02.006783 12312 init.cc:377] Logged in from keytab as kudu/127.21.51.190@KRBTEST.COM (short username kudu)
I20260809 16:23:02.007244 12312 webserver.cc:460] Webserver started at http://127.21.51.190:45677/ using document root <none> and password file <none>
I20260809 16:23:02.007503 12312 fs_manager.cc:359] Metadata directory not provided
I20260809 16:23:02.007614 12312 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:23:02.010593 12312 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.000s sys 0.003s
I20260809 16:23:02.011287 12340 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:23:02.011473 12312 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 16:23:02.011583 12312 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292417722232-21710-0/minicluster-data/master-0/data,/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292417722232-21710-0/minicluster-data/master-0/wal
uuid: "3b5cfde595b240be93937bb3ae6a7190"
format_stamp: "Formatted at 2026-08-09 16:23:00 on dist-test-slave-s11t"
server_key: "426457e3792cbbc285f1a9de711bb57f"
server_key_iv: "a6f0c5a824caf21e65252a259d7b5892"
server_key_version: "encryptionkey@0"
I20260809 16:23:02.011940 12312 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292417722232-21710-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292417722232-21710-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292417722232-21710-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:23:02.030319 12312 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:23:02.032131 12312 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:23:02.032297 12312 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:23:02.036727 12312 rpc_server.cc:307] RPC server started. Bound to: 127.21.51.190:36707
I20260809 16:23:02.036870 12416 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.51.190:36707 every 8 connection(s)
I20260809 16:23:02.037210 12312 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292417722232-21710-0/minicluster-data/master-0/data/info.pb
I20260809 16:23:02.038838 12417 sys_catalog.cc:263] Verifying existing consensus state
I20260809 16:23:02.039515 12417 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3b5cfde595b240be93937bb3ae6a7190: Bootstrap starting.
I20260809 16:23:02.046358 21710 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrNqUCn/build/release/bin/kudu as pid 12312
I20260809 16:23:02.046681 21710 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrNqUCn/build/release/bin/kudu
/tmp/dist-test-taskrNqUCn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292417722232-21710-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292417722232-21710-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292417722232-21710-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292417722232-21710-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.51.129:41875
--local_ip_for_outbound_sockets=127.21.51.129
--tserver_master_addrs=127.21.51.190:36707
--webserver_port=43581
--webserver_interface=127.21.51.129
--keytab_file=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.129.keytab
--principal=kudu/127.21.51.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.51.148:40989
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292417722232-21710-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292417722232-21710-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292417722232-21710-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 16:23:02.048066 12417 log.cc:824] T 00000000000000000000000000000000 P 3b5cfde595b240be93937bb3ae6a7190: Log is configured to *not* fsync() on all Append() calls
I20260809 16:23:02.051379 12417 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3b5cfde595b240be93937bb3ae6a7190: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 16:23:02.051733 12417 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3b5cfde595b240be93937bb3ae6a7190: Bootstrap complete.
I20260809 16:23:02.055897 12417 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3b5cfde595b240be93937bb3ae6a7190 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b5cfde595b240be93937bb3ae6a7190" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 36707 } }
I20260809 16:23:02.056304 12417 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3b5cfde595b240be93937bb3ae6a7190 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b5cfde595b240be93937bb3ae6a7190, State: Initialized, Role: FOLLOWER
I20260809 16:23:02.056496 12417 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3b5cfde595b240be93937bb3ae6a7190 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b5cfde595b240be93937bb3ae6a7190" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 36707 } }
I20260809 16:23:02.056586 12417 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3b5cfde595b240be93937bb3ae6a7190 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 16:23:02.056648 12417 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3b5cfde595b240be93937bb3ae6a7190 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 16:23:02.056727 12417 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3b5cfde595b240be93937bb3ae6a7190 [term 2 FOLLOWER]: Advancing to term 3
I20260809 16:23:02.058323 12417 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3b5cfde595b240be93937bb3ae6a7190 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b5cfde595b240be93937bb3ae6a7190" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 36707 } }
I20260809 16:23:02.058456 12417 leader_election.cc:302] T 00000000000000000000000000000000 P 3b5cfde595b240be93937bb3ae6a7190 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3b5cfde595b240be93937bb3ae6a7190; no voters:
I20260809 16:23:02.058645 12417 leader_election.cc:288] T 00000000000000000000000000000000 P 3b5cfde595b240be93937bb3ae6a7190 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260809 16:23:02.058723 12421 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3b5cfde595b240be93937bb3ae6a7190 [term 3 FOLLOWER]: Leader election won for term 3
I20260809 16:23:02.058887 12421 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3b5cfde595b240be93937bb3ae6a7190 [term 3 LEADER]: Becoming Leader. State: Replica: 3b5cfde595b240be93937bb3ae6a7190, State: Running, Role: LEADER
I20260809 16:23:02.058946 12417 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3b5cfde595b240be93937bb3ae6a7190 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 16:23:02.058986 12421 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3b5cfde595b240be93937bb3ae6a7190 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b5cfde595b240be93937bb3ae6a7190" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 36707 } }
I20260809 16:23:02.059228 12421 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3b5cfde595b240be93937bb3ae6a7190 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3b5cfde595b240be93937bb3ae6a7190. Latest consensus state: current_term: 3 leader_uuid: "3b5cfde595b240be93937bb3ae6a7190" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b5cfde595b240be93937bb3ae6a7190" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 36707 } } }
I20260809 16:23:02.059284 12421 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3b5cfde595b240be93937bb3ae6a7190 [sys.catalog]: This master's current role is: LEADER
I20260809 16:23:02.059530 12422 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3b5cfde595b240be93937bb3ae6a7190 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "3b5cfde595b240be93937bb3ae6a7190" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b5cfde595b240be93937bb3ae6a7190" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 36707 } } }
I20260809 16:23:02.059636 12422 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3b5cfde595b240be93937bb3ae6a7190 [sys.catalog]: This master's current role is: LEADER
I20260809 16:23:02.060148 12430 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 16:23:02.061010 12430 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 16:23:02.061136 12430 catalog_manager.cc:1259] Loaded cluster ID: 0c67fa9120874d83b449e4f732e4ad1a
I20260809 16:23:02.061360 12430 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260809 16:23:02.061749 12430 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f6340ec19ad google::LogMessage::Fail() at ??:0
@ 0x7f6340ec488a google::LogMessage::SendToLog() at ??:0
@ 0x7f6340ec14de google::LogMessage::Flush() at ??:0
@ 0x7f6340ec2a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f6348de60f1 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7f6348def8b9 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f6342d8f0d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f6342d84954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f6343be86db start_thread at ??:0
@ 0x7f633f9bf61f clone at ??:0
W20260809 16:23:02.144873 12419 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:23:02.145087 12419 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:23:02.145124 12419 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:23:02.146247 12419 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:23:02.146307 12419 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:23:02.146353 12419 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.51.129
I20260809 16:23:02.147846 12419 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.51.148:40989
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292417722232-21710-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292417722232-21710-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.129.keytab
--principal=kudu/127.21.51.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.51.129:41875
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292417722232-21710-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.51.129
--webserver_port=43581
--tserver_master_addrs=127.21.51.190:36707
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12419
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.51.129
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292417722232-21710-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:23:02.148120 12419 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:23:02.148355 12419 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:23:02.153378 12472 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:23:02.153388 12476 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:23:02.153403 12471 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:23:02.153630 12419 server_base.cc:1034] running on GCE node
I20260809 16:23:02.153838 12419 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:23:02.154105 12419 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:23:02.155292 12419 hybrid_clock.cc:648] HybridClock initialized: now 1786292582155277 us; error 27 us; skew 500 ppm
Aug 09 16:23:02 dist-test-slave-s11t krb5kdc[10712](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292582, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.51.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:23:02.158329 12419 init.cc:377] Logged in from keytab as kudu/127.21.51.129@KRBTEST.COM (short username kudu)
I20260809 16:23:02.158742 12419 webserver.cc:460] Webserver started at http://127.21.51.129:43581/ using document root <none> and password file <none>
I20260809 16:23:02.158994 12419 fs_manager.cc:359] Metadata directory not provided
I20260809 16:23:02.159065 12419 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:23:02.160436 12419 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:23:02.161295 12484 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:23:02.161470 12419 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260809 16:23:02.161568 12419 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292417722232-21710-0/minicluster-data/ts-0/data,/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292417722232-21710-0/minicluster-data/ts-0/wal
uuid: "57576785ec93441f9b827f3a8b6df2ed"
format_stamp: "Formatted at 2026-08-09 16:23:00 on dist-test-slave-s11t"
server_key: "47e6daf2d4b2c8ffbb5ffec74976c636"
server_key_iv: "acafa1b1889fe54c341da0eec10112e2"
server_key_version: "encryptionkey@0"
I20260809 16:23:02.161931 12419 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292417722232-21710-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292417722232-21710-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292417722232-21710-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:23:02.173218 12419 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:23:02.175390 12419 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:23:02.175546 12419 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:23:02.175767 12419 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:23:02.176225 12497 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260809 16:23:02.178192 12419 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260809 16:23:02.178251 12419 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260809 16:23:02.178306 12419 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260809 16:23:02.178934 12419 ts_tablet_manager.cc:610] Registered 1 tablets
I20260809 16:23:02.178988 12419 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260809 16:23:02.179064 12497 tablet_bootstrap.cc:492] T 8e7536df343f4e51853cff79a145f428 P 57576785ec93441f9b827f3a8b6df2ed: Bootstrap starting.
I20260809 16:23:02.180951 12497 log.cc:824] T 8e7536df343f4e51853cff79a145f428 P 57576785ec93441f9b827f3a8b6df2ed: Log is configured to *not* fsync() on all Append() calls
I20260809 16:23:02.183880 12497 tablet_bootstrap.cc:492] T 8e7536df343f4e51853cff79a145f428 P 57576785ec93441f9b827f3a8b6df2ed: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 16:23:02.184196 12497 tablet_bootstrap.cc:492] T 8e7536df343f4e51853cff79a145f428 P 57576785ec93441f9b827f3a8b6df2ed: Bootstrap complete.
I20260809 16:23:02.184365 12497 ts_tablet_manager.cc:1397] T 8e7536df343f4e51853cff79a145f428 P 57576785ec93441f9b827f3a8b6df2ed: Time spent bootstrapping tablet: real 0.005s user 0.005s sys 0.000s
I20260809 16:23:02.186427 12419 rpc_server.cc:307] RPC server started. Bound to: 127.21.51.129:41875
I20260809 16:23:02.186766 12419 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292417722232-21710-0/minicluster-data/ts-0/data/info.pb
I20260809 16:23:02.186691 12497 raft_consensus.cc:348] T 8e7536df343f4e51853cff79a145f428 P 57576785ec93441f9b827f3a8b6df2ed [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "274512f1889a4254b485fef854967321" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 42799 } } peers { permanent_uuid: "5ba1dc75657f4418b6903fa6ceed3d8b" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 46397 } } peers { permanent_uuid: "57576785ec93441f9b827f3a8b6df2ed" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 41875 } }
I20260809 16:23:02.187258 12497 raft_consensus.cc:729] T 8e7536df343f4e51853cff79a145f428 P 57576785ec93441f9b827f3a8b6df2ed [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57576785ec93441f9b827f3a8b6df2ed, State: Initialized, Role: FOLLOWER
I20260809 16:23:02.187449 12497 consensus_queue.cc:260] T 8e7536df343f4e51853cff79a145f428 P 57576785ec93441f9b827f3a8b6df2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "274512f1889a4254b485fef854967321" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 42799 } } peers { permanent_uuid: "5ba1dc75657f4418b6903fa6ceed3d8b" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 46397 } } peers { permanent_uuid: "57576785ec93441f9b827f3a8b6df2ed" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 41875 } }
I20260809 16:23:02.188134 12664 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.51.129:41875 every 8 connection(s)
I20260809 16:23:02.188149 12497 ts_tablet_manager.cc:1428] T 8e7536df343f4e51853cff79a145f428 P 57576785ec93441f9b827f3a8b6df2ed: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260809 16:23:02.193783 21710 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrNqUCn/build/release/bin/kudu as pid 12419
I20260809 16:23:02.194072 21710 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrNqUCn/build/release/bin/kudu
/tmp/dist-test-taskrNqUCn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292417722232-21710-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292417722232-21710-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292417722232-21710-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292417722232-21710-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.51.130:46397
--local_ip_for_outbound_sockets=127.21.51.130
--tserver_master_addrs=127.21.51.190:36707
--webserver_port=45661
--webserver_interface=127.21.51.130
--keytab_file=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.130.keytab
--principal=kudu/127.21.51.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.51.148:40989
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292417722232-21710-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292417722232-21710-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292417722232-21710-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:23:02.279186 12673 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:23:02.188856 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:36707 (local address 127.21.51.129:40023)
0809 16:23:02.189014 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0809 16:23:02.189047 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0809 16:23:02.189165 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:23:02.278959 (+ 89794us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.51.190:36707: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":81,"threads_started":1}
W20260809 16:23:02.279419 12668 heartbeater.cc:644] Failed to heartbeat to 127.21.51.190:36707 (0 consecutive failures): Network error: Failed to ping master at 127.21.51.190:36707: Client connection negotiation failed: client connection to 127.21.51.190:36707: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260809 16:23:02.279783 12673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:23:02.279628 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:36707 (local address 127.21.51.129:56079)
0809 16:23:02.279692 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0809 16:23:02.279752 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.51.190:36707: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260809 16:23:02.280102 12673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:23:02.280021 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:36707 (local address 127.21.51.129:40547)
0809 16:23:02.280051 (+ 30us) negotiation.cc:107] Waiting for socket to connect
0809 16:23:02.280080 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.51.190:36707: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":16}
W20260809 16:23:02.280223 12668 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260809 16:23:02.286377 12675 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:23:02.286552 12675 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:23:02.286602 12675 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:23:02.287710 12675 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:23:02.287750 12675 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:23:02.287806 12675 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.51.130
I20260809 16:23:02.289311 12675 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.51.148:40989
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292417722232-21710-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292417722232-21710-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.130.keytab
--principal=kudu/127.21.51.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.51.130:46397
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292417722232-21710-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.51.130
--webserver_port=45661
--tserver_master_addrs=127.21.51.190:36707
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12675
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.51.130
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292417722232-21710-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:23:02.289535 12675 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:23:02.289755 12675 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:23:02.294554 12681 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:23:02.294541 12685 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:23:02.294567 12675 server_base.cc:1034] running on GCE node
W20260809 16:23:02.294541 12683 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:23:02.294945 12675 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:23:02.295189 12675 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:23:02.296353 12675 hybrid_clock.cc:648] HybridClock initialized: now 1786292582296345 us; error 33 us; skew 500 ppm
Aug 09 16:23:02 dist-test-slave-s11t krb5kdc[10712](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292582, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.51.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:23:02.299007 12675 init.cc:377] Logged in from keytab as kudu/127.21.51.130@KRBTEST.COM (short username kudu)
I20260809 16:23:02.299422 12675 webserver.cc:460] Webserver started at http://127.21.51.130:45661/ using document root <none> and password file <none>
I20260809 16:23:02.299652 12675 fs_manager.cc:359] Metadata directory not provided
I20260809 16:23:02.299721 12675 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:23:02.301065 12675 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.001s
I20260809 16:23:02.301649 12698 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:23:02.301812 12675 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 16:23:02.301901 12675 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292417722232-21710-0/minicluster-data/ts-1/data,/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292417722232-21710-0/minicluster-data/ts-1/wal
uuid: "5ba1dc75657f4418b6903fa6ceed3d8b"
format_stamp: "Formatted at 2026-08-09 16:23:00 on dist-test-slave-s11t"
server_key: "b1aee06ad35f0c32fe8b3fc4df3cfac2"
server_key_iv: "42725d4f5393d5976a7fc58d89e13374"
server_key_version: "encryptionkey@0"
I20260809 16:23:02.302220 12675 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292417722232-21710-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292417722232-21710-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292417722232-21710-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:23:02.310703 12675 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:23:02.312969 12675 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:23:02.313123 12675 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:23:02.313338 12675 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:23:02.313648 12675 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:23:02.313701 12675 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:23:02.313742 12675 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:23:02.313766 12675 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:23:02.319650 12675 rpc_server.cc:307] RPC server started. Bound to: 127.21.51.130:46397
I20260809 16:23:02.319702 12873 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.51.130:46397 every 8 connection(s)
I20260809 16:23:02.320021 12675 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292417722232-21710-0/minicluster-data/ts-1/data/info.pb
I20260809 16:23:02.320513 21710 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrNqUCn/build/release/bin/kudu as pid 12675
I20260809 16:23:02.320869 21710 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrNqUCn/build/release/bin/kudu
/tmp/dist-test-taskrNqUCn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292417722232-21710-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292417722232-21710-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292417722232-21710-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292417722232-21710-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.51.131:42799
--local_ip_for_outbound_sockets=127.21.51.131
--tserver_master_addrs=127.21.51.190:36707
--webserver_port=38313
--webserver_interface=127.21.51.131
--keytab_file=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.131.keytab
--principal=kudu/127.21.51.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.51.148:40989
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292417722232-21710-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292417722232-21710-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292417722232-21710-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 16:23:02.321198 12878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:23:02.320717 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:36707 (local address 127.21.51.130:46849)
0809 16:23:02.320865 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0809 16:23:02.320982 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.51.190:36707: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":83,"threads_started":1}
W20260809 16:23:02.321595 12875 heartbeater.cc:644] Failed to heartbeat to 127.21.51.190:36707 (0 consecutive failures): Network error: Failed to ping master at 127.21.51.190:36707: Client connection negotiation failed: client connection to 127.21.51.190:36707: connect: Connection refused (error 111)
I20260809 16:23:02.321909 12878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:23:02.321806 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:36707 (local address 127.21.51.130:39673)
0809 16:23:02.321854 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0809 16:23:02.321879 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.51.190:36707: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260809 16:23:02.322203 12878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:23:02.322117 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:36707 (local address 127.21.51.130:44441)
0809 16:23:02.322161 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0809 16:23:02.322184 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.51.190:36707: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260809 16:23:02.322310 12875 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260809 16:23:02.403447 12879 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:23:02.403671 12879 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:23:02.403709 12879 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:23:02.404899 12879 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:23:02.404961 12879 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:23:02.405007 12879 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.51.131
I20260809 16:23:02.406520 12879 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.51.148:40989
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292417722232-21710-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292417722232-21710-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.131.keytab
--principal=kudu/127.21.51.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.51.131:42799
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292417722232-21710-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.51.131
--webserver_port=38313
--tserver_master_addrs=127.21.51.190:36707
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12879
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.51.131
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292417722232-21710-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:23:02.406778 12879 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:23:02.407008 12879 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:23:02.412021 12896 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:23:02.412017 12890 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:23:02.412226 12879 server_base.cc:1034] running on GCE node
W20260809 16:23:02.412097 12891 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:23:02.412513 12879 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:23:02.412827 12879 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:23:02.414011 12879 hybrid_clock.cc:648] HybridClock initialized: now 1786292582414011 us; error 43 us; skew 500 ppm
Aug 09 16:23:02 dist-test-slave-s11t krb5kdc[10712](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292582, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.51.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:23:02.416628 12879 init.cc:377] Logged in from keytab as kudu/127.21.51.131@KRBTEST.COM (short username kudu)
I20260809 16:23:02.417111 12879 webserver.cc:460] Webserver started at http://127.21.51.131:38313/ using document root <none> and password file <none>
I20260809 16:23:02.417343 12879 fs_manager.cc:359] Metadata directory not provided
I20260809 16:23:02.417390 12879 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:23:02.418670 12879 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:23:02.419251 12903 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:23:02.419445 12879 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 16:23:02.419545 12879 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292417722232-21710-0/minicluster-data/ts-2/data,/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292417722232-21710-0/minicluster-data/ts-2/wal
uuid: "274512f1889a4254b485fef854967321"
format_stamp: "Formatted at 2026-08-09 16:23:00 on dist-test-slave-s11t"
server_key: "5c0923b2d7f6c75f47c908daa014db10"
server_key_iv: "9d15cee54955387cc133adf55e93d3fc"
server_key_version: "encryptionkey@0"
I20260809 16:23:02.419864 12879 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292417722232-21710-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292417722232-21710-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292417722232-21710-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:23:02.432272 12879 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:23:02.434518 12879 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:23:02.434684 12879 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:23:02.434904 12879 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:23:02.435206 12879 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:23:02.435261 12879 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:23:02.435294 12879 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:23:02.435321 12879 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:23:02.441425 12879 rpc_server.cc:307] RPC server started. Bound to: 127.21.51.131:42799
I20260809 16:23:02.441479 13074 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.51.131:42799 every 8 connection(s)
I20260809 16:23:02.441820 12879 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292417722232-21710-0/minicluster-data/ts-2/data/info.pb
I20260809 16:23:02.442839 13077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:23:02.442447 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:36707 (local address 127.21.51.131:42337)
0809 16:23:02.442588 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0809 16:23:02.442704 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.51.190:36707: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":69,"threads_started":1}
W20260809 16:23:02.443214 13075 heartbeater.cc:644] Failed to heartbeat to 127.21.51.190:36707 (0 consecutive failures): Network error: Failed to ping master at 127.21.51.190:36707: Client connection negotiation failed: client connection to 127.21.51.190:36707: connect: Connection refused (error 111)
I20260809 16:23:02.443511 13077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:23:02.443389 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:36707 (local address 127.21.51.131:54373)
0809 16:23:02.443460 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0809 16:23:02.443490 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.51.190:36707: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260809 16:23:02.443778 13077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:23:02.443722 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:36707 (local address 127.21.51.131:38137)
0809 16:23:02.443744 (+ 22us) negotiation.cc:107] Waiting for socket to connect
0809 16:23:02.443763 (+ 19us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.51.190:36707: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":12}
W20260809 16:23:02.443892 13075 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260809 16:23:02.446419 21710 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrNqUCn/build/release/bin/kudu as pid 12879
I20260809 16:23:03.280879 13079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:23:03.280580 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:36707 (local address 127.21.51.129:48453)
0809 16:23:03.280770 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0809 16:23:03.280809 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.51.190:36707: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":70,"threads_started":1}
I20260809 16:23:03.322825 13080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:23:03.322608 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:36707 (local address 127.21.51.130:58141)
0809 16:23:03.322741 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0809 16:23:03.322777 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.51.190:36707: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":58,"threads_started":1}
I20260809 16:23:03.408651 13081 raft_consensus.cc:482] T 8e7536df343f4e51853cff79a145f428 P 57576785ec93441f9b827f3a8b6df2ed [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 16:23:03.408900 13081 raft_consensus.cc:504] T 8e7536df343f4e51853cff79a145f428 P 57576785ec93441f9b827f3a8b6df2ed [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "274512f1889a4254b485fef854967321" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 42799 } } peers { permanent_uuid: "5ba1dc75657f4418b6903fa6ceed3d8b" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 46397 } } peers { permanent_uuid: "57576785ec93441f9b827f3a8b6df2ed" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 41875 } }
I20260809 16:23:03.409293 13081 leader_election.cc:288] T 8e7536df343f4e51853cff79a145f428 P 57576785ec93441f9b827f3a8b6df2ed [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 274512f1889a4254b485fef854967321 (127.21.51.131:42799), 5ba1dc75657f4418b6903fa6ceed3d8b (127.21.51.130:46397)
Aug 09 16:23:03 dist-test-slave-s11t krb5kdc[10712](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292582, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.51.129@KRBTEST.COM for kudu/127.21.51.131@KRBTEST.COM
Aug 09 16:23:03 dist-test-slave-s11t krb5kdc[10712](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292582, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.51.129@KRBTEST.COM for kudu/127.21.51.130@KRBTEST.COM
I20260809 16:23:03.421446 13086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:23:03.409755 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.51.129:54019 (local address 127.21.51.130:46397)
0809 16:23:03.409904 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0809 16:23:03.409912 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 16:23:03.409946 (+ 34us) server_negotiation.cc:407] Connection header received
0809 16:23:03.410367 (+ 421us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:23:03.410368 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:23:03.410407 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:23:03.410563 (+ 156us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:23:03.411362 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:23:03.412239 (+ 877us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:23:03.413769 (+ 1530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:23:03.414407 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:23:03.414529 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:23:03.417840 (+ 3311us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:23:03.417848 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:23:03.417849 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:23:03.417859 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:23:03.419684 (+ 1825us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:23:03.420130 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:23:03.420132 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:23:03.420133 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:23:03.420216 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:23:03.420496 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:23:03.420497 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:23:03.420498 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:23:03.420816 (+ 318us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:23:03.420907 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0809 16:23:03.421193 (+ 286us) server_negotiation.cc:299] Negotiation successful
0809 16:23:03.421323 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":108,"thread_start_us":77,"threads_started":1}
I20260809 16:23:03.421450 13079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:23:03.409501 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.131:42799 (local address 127.21.51.129:42961)
0809 16:23:03.409753 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0809 16:23:03.409766 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 16:23:03.409878 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:23:03.410604 (+ 726us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:23:03.410607 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:23:03.411010 (+ 403us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:23:03.411426 (+ 416us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:23:03.411429 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:23:03.412628 (+ 1199us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:23:03.412628 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:23:03.413666 (+ 1038us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:23:03.413669 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:23:03.414523 (+ 854us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:23:03.414525 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:23:03.414677 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:23:03.415346 (+ 669us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:23:03.415358 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:23:03.417337 (+ 1979us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:23:03.419621 (+ 2284us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:23:03.419629 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:23:03.419631 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:23:03.420011 (+ 380us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:23:03.420280 (+ 269us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:23:03.420281 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:23:03.420281 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:23:03.420395 (+ 114us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:23:03.420913 (+ 518us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:23:03.420914 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:23:03.421025 (+ 111us) client_negotiation.cc:769] Sending connection context
0809 16:23:03.421132 (+ 107us) client_negotiation.cc:240] Negotiation successful
0809 16:23:03.421335 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":44,"mutex_wait_us":32}
I20260809 16:23:03.421479 13084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:23:03.409606 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.51.129:42961 (local address 127.21.51.131:42799)
0809 16:23:03.409751 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0809 16:23:03.409765 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0809 16:23:03.409879 (+ 114us) server_negotiation.cc:407] Connection header received
0809 16:23:03.410361 (+ 482us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:23:03.410364 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:23:03.410440 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:23:03.410591 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:23:03.411559 (+ 968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:23:03.412526 (+ 967us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:23:03.413769 (+ 1243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:23:03.414405 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:23:03.414535 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:23:03.417516 (+ 2981us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:23:03.417521 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:23:03.417521 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:23:03.417536 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:23:03.419531 (+ 1995us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:23:03.420099 (+ 568us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:23:03.420102 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:23:03.420103 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:23:03.420206 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:23:03.420493 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:23:03.420494 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:23:03.420494 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:23:03.420820 (+ 326us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:23:03.420920 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0809 16:23:03.421206 (+ 286us) server_negotiation.cc:299] Negotiation successful
0809 16:23:03.421372 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":107,"thread_start_us":85,"threads_started":1}
I20260809 16:23:03.422186 13083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:23:03.409554 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.130:46397 (local address 127.21.51.129:54019)
0809 16:23:03.409680 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0809 16:23:03.409694 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 16:23:03.409756 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:23:03.410604 (+ 848us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:23:03.410607 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:23:03.411010 (+ 403us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:23:03.411236 (+ 226us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:23:03.411241 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:23:03.412358 (+ 1117us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:23:03.412359 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:23:03.413666 (+ 1307us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:23:03.413669 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:23:03.414525 (+ 856us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:23:03.414526 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:23:03.414677 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:23:03.415346 (+ 669us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:23:03.415358 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:23:03.417728 (+ 2370us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:23:03.419784 (+ 2056us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:23:03.419785 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:23:03.419785 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:23:03.420027 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:23:03.420311 (+ 284us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:23:03.420312 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:23:03.420312 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:23:03.420398 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:23:03.420888 (+ 490us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:23:03.420896 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:23:03.421025 (+ 129us) client_negotiation.cc:769] Sending connection context
0809 16:23:03.421133 (+ 108us) client_negotiation.cc:240] Negotiation successful
0809 16:23:03.421419 (+ 286us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":109,"mutex_wait_us":144,"thread_start_us":56,"threads_started":1}
W20260809 16:23:03.422358 12485 leader_election.cc:341] T 8e7536df343f4e51853cff79a145f428 P 57576785ec93441f9b827f3a8b6df2ed [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 274512f1889a4254b485fef854967321 (127.21.51.131:42799): Not found: Tablet not found: 8e7536df343f4e51853cff79a145f428
W20260809 16:23:03.422878 12492 leader_election.cc:341] T 8e7536df343f4e51853cff79a145f428 P 57576785ec93441f9b827f3a8b6df2ed [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 5ba1dc75657f4418b6903fa6ceed3d8b (127.21.51.130:46397): Not found: Tablet not found: 8e7536df343f4e51853cff79a145f428
I20260809 16:23:03.422930 12492 leader_election.cc:302] T 8e7536df343f4e51853cff79a145f428 P 57576785ec93441f9b827f3a8b6df2ed [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57576785ec93441f9b827f3a8b6df2ed; no voters: 274512f1889a4254b485fef854967321, 5ba1dc75657f4418b6903fa6ceed3d8b
I20260809 16:23:03.423026 13081 raft_consensus.cc:2726] T 8e7536df343f4e51853cff79a145f428 P 57576785ec93441f9b827f3a8b6df2ed [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 16:23:03.444578 13087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:23:03.444285 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:36707 (local address 127.21.51.131:36149)
0809 16:23:03.444469 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0809 16:23:03.444528 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.51.190:36707: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":106,"threads_started":1}
I20260809 16:23:04.281571 13088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:23:04.281352 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:36707 (local address 127.21.51.129:55123)
0809 16:23:04.281486 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0809 16:23:04.281523 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.51.190:36707: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":70,"threads_started":1}
I20260809 16:23:04.323514 13089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:23:04.323276 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:36707 (local address 127.21.51.130:58289)
0809 16:23:04.323430 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0809 16:23:04.323474 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.51.190:36707: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":64,"threads_started":1}
I20260809 16:23:04.445186 13092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:23:04.444999 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:36707 (local address 127.21.51.131:34077)
0809 16:23:04.445117 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0809 16:23:04.445151 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.51.190:36707: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":62,"threads_started":1}
I20260809 16:23:05.282249 13093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:23:05.282029 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:36707 (local address 127.21.51.129:39295)
0809 16:23:05.282164 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0809 16:23:05.282201 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.51.190:36707: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":66,"threads_started":1}
I20260809 16:23:05.324155 13094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:23:05.323953 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:36707 (local address 127.21.51.130:33447)
0809 16:23:05.324079 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0809 16:23:05.324114 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.51.190:36707: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":62,"threads_started":1}
I20260809 16:23:05.398072 13096 raft_consensus.cc:482] T 8e7536df343f4e51853cff79a145f428 P 57576785ec93441f9b827f3a8b6df2ed [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 16:23:05.398193 13096 raft_consensus.cc:504] T 8e7536df343f4e51853cff79a145f428 P 57576785ec93441f9b827f3a8b6df2ed [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "274512f1889a4254b485fef854967321" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 42799 } } peers { permanent_uuid: "5ba1dc75657f4418b6903fa6ceed3d8b" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 46397 } } peers { permanent_uuid: "57576785ec93441f9b827f3a8b6df2ed" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 41875 } }
I20260809 16:23:05.398412 13096 leader_election.cc:288] T 8e7536df343f4e51853cff79a145f428 P 57576785ec93441f9b827f3a8b6df2ed [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 274512f1889a4254b485fef854967321 (127.21.51.131:42799), 5ba1dc75657f4418b6903fa6ceed3d8b (127.21.51.130:46397)
W20260809 16:23:05.398869 12485 leader_election.cc:341] T 8e7536df343f4e51853cff79a145f428 P 57576785ec93441f9b827f3a8b6df2ed [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 274512f1889a4254b485fef854967321 (127.21.51.131:42799): Not found: Tablet not found: 8e7536df343f4e51853cff79a145f428
W20260809 16:23:05.398947 12492 leader_election.cc:341] T 8e7536df343f4e51853cff79a145f428 P 57576785ec93441f9b827f3a8b6df2ed [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 5ba1dc75657f4418b6903fa6ceed3d8b (127.21.51.130:46397): Not found: Tablet not found: 8e7536df343f4e51853cff79a145f428
I20260809 16:23:05.399026 12492 leader_election.cc:302] T 8e7536df343f4e51853cff79a145f428 P 57576785ec93441f9b827f3a8b6df2ed [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57576785ec93441f9b827f3a8b6df2ed; no voters: 274512f1889a4254b485fef854967321, 5ba1dc75657f4418b6903fa6ceed3d8b
I20260809 16:23:05.399118 13096 raft_consensus.cc:2726] T 8e7536df343f4e51853cff79a145f428 P 57576785ec93441f9b827f3a8b6df2ed [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 16:23:05.445935 13098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:23:05.445700 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:36707 (local address 127.21.51.131:51383)
0809 16:23:05.445851 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0809 16:23:05.445890 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.51.190:36707: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":86,"threads_started":1}
I20260809 16:23:06.283056 13100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:23:06.282813 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:36707 (local address 127.21.51.129:35673)
0809 16:23:06.282965 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0809 16:23:06.283009 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.51.190:36707: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":68,"threads_started":1}
I20260809 16:23:06.324892 13101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:23:06.324613 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:36707 (local address 127.21.51.130:51079)
0809 16:23:06.324810 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0809 16:23:06.324851 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.51.190:36707: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":116,"threads_started":1}
I20260809 16:23:06.446722 13103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:23:06.446464 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:36707 (local address 127.21.51.131:33571)
0809 16:23:06.446635 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0809 16:23:06.446675 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.51.190:36707: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":83,"threads_started":1}
I20260809 16:23:07.283860 13104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:23:07.283586 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:36707 (local address 127.21.51.129:46105)
0809 16:23:07.283774 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0809 16:23:07.283814 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.51.190:36707: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":85,"threads_started":1}
I20260809 16:23:07.325580 13105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:23:07.325365 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:36707 (local address 127.21.51.130:45919)
0809 16:23:07.325503 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0809 16:23:07.325541 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.51.190:36707: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":78,"threads_started":1}
I20260809 16:23:07.447388 13106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:23:07.447170 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:36707 (local address 127.21.51.131:54307)
0809 16:23:07.447313 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0809 16:23:07.447350 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.51.190:36707: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":80,"threads_started":1}
I20260809 16:23:07.776895 13107 raft_consensus.cc:482] T 8e7536df343f4e51853cff79a145f428 P 57576785ec93441f9b827f3a8b6df2ed [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 16:23:07.776993 13107 raft_consensus.cc:504] T 8e7536df343f4e51853cff79a145f428 P 57576785ec93441f9b827f3a8b6df2ed [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "274512f1889a4254b485fef854967321" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 42799 } } peers { permanent_uuid: "5ba1dc75657f4418b6903fa6ceed3d8b" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 46397 } } peers { permanent_uuid: "57576785ec93441f9b827f3a8b6df2ed" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 41875 } }
I20260809 16:23:07.777318 13107 leader_election.cc:288] T 8e7536df343f4e51853cff79a145f428 P 57576785ec93441f9b827f3a8b6df2ed [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 274512f1889a4254b485fef854967321 (127.21.51.131:42799), 5ba1dc75657f4418b6903fa6ceed3d8b (127.21.51.130:46397)
W20260809 16:23:07.777622 12485 leader_election.cc:341] T 8e7536df343f4e51853cff79a145f428 P 57576785ec93441f9b827f3a8b6df2ed [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 274512f1889a4254b485fef854967321 (127.21.51.131:42799): Not found: Tablet not found: 8e7536df343f4e51853cff79a145f428
W20260809 16:23:07.777694 12492 leader_election.cc:341] T 8e7536df343f4e51853cff79a145f428 P 57576785ec93441f9b827f3a8b6df2ed [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 5ba1dc75657f4418b6903fa6ceed3d8b (127.21.51.130:46397): Not found: Tablet not found: 8e7536df343f4e51853cff79a145f428
I20260809 16:23:07.777730 12492 leader_election.cc:302] T 8e7536df343f4e51853cff79a145f428 P 57576785ec93441f9b827f3a8b6df2ed [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57576785ec93441f9b827f3a8b6df2ed; no voters: 274512f1889a4254b485fef854967321, 5ba1dc75657f4418b6903fa6ceed3d8b
I20260809 16:23:07.777820 13107 raft_consensus.cc:2726] T 8e7536df343f4e51853cff79a145f428 P 57576785ec93441f9b827f3a8b6df2ed [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 16:23:08.284672 13109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:23:08.284437 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:36707 (local address 127.21.51.129:50671)
0809 16:23:08.284579 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0809 16:23:08.284623 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.51.190:36707: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":68,"threads_started":1}
I20260809 16:23:08.326400 13110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:23:08.326147 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:36707 (local address 127.21.51.130:51353)
0809 16:23:08.326311 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0809 16:23:08.326354 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.51.190:36707: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":85,"threads_started":1}
I20260809 16:23:08.448160 13112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:23:08.447925 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:36707 (local address 127.21.51.131:52631)
0809 16:23:08.448063 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0809 16:23:08.448112 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.51.190:36707: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":67,"threads_started":1}
I20260809 16:23:09.285547 13113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:23:09.285285 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:36707 (local address 127.21.51.129:46135)
0809 16:23:09.285455 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0809 16:23:09.285490 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.51.190:36707: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":120,"threads_started":1}
I20260809 16:23:09.327131 13115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:23:09.326893 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:36707 (local address 127.21.51.130:37903)
0809 16:23:09.327056 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0809 16:23:09.327093 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.51.190:36707: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":80,"threads_started":1}
I20260809 16:23:09.449038 13117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:23:09.448764 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:36707 (local address 127.21.51.131:39087)
0809 16:23:09.448952 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0809 16:23:09.448991 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.51.190:36707: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":86,"threads_started":1}
I20260809 16:23:10.286355 13119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:23:10.286106 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:36707 (local address 127.21.51.129:41881)
0809 16:23:10.286262 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0809 16:23:10.286304 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.51.190:36707: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":70,"threads_started":1}
I20260809 16:23:10.327881 13120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:23:10.327641 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:36707 (local address 127.21.51.130:52395)
0809 16:23:10.327792 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0809 16:23:10.327839 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.51.190:36707: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":64,"threads_started":1}
I20260809 16:23:10.449695 13121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:23:10.449474 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:36707 (local address 127.21.51.131:55013)
0809 16:23:10.449609 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0809 16:23:10.449647 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.51.190:36707: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":61,"threads_started":1}
I20260809 16:23:11.287130 13122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:23:11.286866 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:36707 (local address 127.21.51.129:53739)
0809 16:23:11.287041 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0809 16:23:11.287081 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.51.190:36707: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":100,"threads_started":1}
I20260809 16:23:11.328590 13123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:23:11.328342 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:36707 (local address 127.21.51.130:44433)
0809 16:23:11.328504 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0809 16:23:11.328545 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.51.190:36707: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":86,"threads_started":1}
I20260809 16:23:11.450390 13124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:23:11.450173 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:36707 (local address 127.21.51.131:48185)
0809 16:23:11.450296 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0809 16:23:11.450331 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.51.190:36707: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":64,"threads_started":1}
I20260809 16:23:11.539563 13125 raft_consensus.cc:482] T 8e7536df343f4e51853cff79a145f428 P 57576785ec93441f9b827f3a8b6df2ed [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 16:23:11.539667 13125 raft_consensus.cc:504] T 8e7536df343f4e51853cff79a145f428 P 57576785ec93441f9b827f3a8b6df2ed [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "274512f1889a4254b485fef854967321" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 42799 } } peers { permanent_uuid: "5ba1dc75657f4418b6903fa6ceed3d8b" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 46397 } } peers { permanent_uuid: "57576785ec93441f9b827f3a8b6df2ed" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 41875 } }
I20260809 16:23:11.539873 13125 leader_election.cc:288] T 8e7536df343f4e51853cff79a145f428 P 57576785ec93441f9b827f3a8b6df2ed [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 274512f1889a4254b485fef854967321 (127.21.51.131:42799), 5ba1dc75657f4418b6903fa6ceed3d8b (127.21.51.130:46397)
W20260809 16:23:11.540272 12492 leader_election.cc:341] T 8e7536df343f4e51853cff79a145f428 P 57576785ec93441f9b827f3a8b6df2ed [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 5ba1dc75657f4418b6903fa6ceed3d8b (127.21.51.130:46397): Not found: Tablet not found: 8e7536df343f4e51853cff79a145f428
W20260809 16:23:11.540349 12485 leader_election.cc:341] T 8e7536df343f4e51853cff79a145f428 P 57576785ec93441f9b827f3a8b6df2ed [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 274512f1889a4254b485fef854967321 (127.21.51.131:42799): Not found: Tablet not found: 8e7536df343f4e51853cff79a145f428
I20260809 16:23:11.540403 12485 leader_election.cc:302] T 8e7536df343f4e51853cff79a145f428 P 57576785ec93441f9b827f3a8b6df2ed [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57576785ec93441f9b827f3a8b6df2ed; no voters: 274512f1889a4254b485fef854967321, 5ba1dc75657f4418b6903fa6ceed3d8b
I20260809 16:23:11.540469 13125 raft_consensus.cc:2726] T 8e7536df343f4e51853cff79a145f428 P 57576785ec93441f9b827f3a8b6df2ed [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 16:23:12.287881 13126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:23:12.287664 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:36707 (local address 127.21.51.129:53005)
0809 16:23:12.287804 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0809 16:23:12.287835 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.51.190:36707: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":89,"threads_started":1}
I20260809 16:23:12.329258 13128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:23:12.329056 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:36707 (local address 127.21.51.130:51627)
0809 16:23:12.329182 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0809 16:23:12.329217 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.51.190:36707: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":69,"threads_started":1}
I20260809 16:23:12.451085 13129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:23:12.450869 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:36707 (local address 127.21.51.131:38869)
0809 16:23:12.451002 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0809 16:23:12.451041 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.51.190:36707: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":68,"threads_started":1}
I20260809 16:23:13.288654 13130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:23:13.288436 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:36707 (local address 127.21.51.129:46269)
0809 16:23:13.288567 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0809 16:23:13.288608 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.51.190:36707: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":70,"threads_started":1}
I20260809 16:23:13.329990 13131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:23:13.329753 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:36707 (local address 127.21.51.130:60097)
0809 16:23:13.329907 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0809 16:23:13.329945 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.51.190:36707: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":90,"threads_started":1}
I20260809 16:23:13.415398 13132 raft_consensus.cc:482] T 8e7536df343f4e51853cff79a145f428 P 57576785ec93441f9b827f3a8b6df2ed [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 16:23:13.415514 13132 raft_consensus.cc:504] T 8e7536df343f4e51853cff79a145f428 P 57576785ec93441f9b827f3a8b6df2ed [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "274512f1889a4254b485fef854967321" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 42799 } } peers { permanent_uuid: "5ba1dc75657f4418b6903fa6ceed3d8b" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 46397 } } peers { permanent_uuid: "57576785ec93441f9b827f3a8b6df2ed" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 41875 } }
I20260809 16:23:13.415684 13132 leader_election.cc:288] T 8e7536df343f4e51853cff79a145f428 P 57576785ec93441f9b827f3a8b6df2ed [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 274512f1889a4254b485fef854967321 (127.21.51.131:42799), 5ba1dc75657f4418b6903fa6ceed3d8b (127.21.51.130:46397)
W20260809 16:23:13.416077 12492 leader_election.cc:341] T 8e7536df343f4e51853cff79a145f428 P 57576785ec93441f9b827f3a8b6df2ed [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 5ba1dc75657f4418b6903fa6ceed3d8b (127.21.51.130:46397): Not found: Tablet not found: 8e7536df343f4e51853cff79a145f428
W20260809 16:23:13.416164 12485 leader_election.cc:341] T 8e7536df343f4e51853cff79a145f428 P 57576785ec93441f9b827f3a8b6df2ed [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 274512f1889a4254b485fef854967321 (127.21.51.131:42799): Not found: Tablet not found: 8e7536df343f4e51853cff79a145f428
I20260809 16:23:13.416271 12485 leader_election.cc:302] T 8e7536df343f4e51853cff79a145f428 P 57576785ec93441f9b827f3a8b6df2ed [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57576785ec93441f9b827f3a8b6df2ed; no voters: 274512f1889a4254b485fef854967321, 5ba1dc75657f4418b6903fa6ceed3d8b
I20260809 16:23:13.416361 13132 raft_consensus.cc:2726] T 8e7536df343f4e51853cff79a145f428 P 57576785ec93441f9b827f3a8b6df2ed [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 16:23:13.451809 13134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:23:13.451558 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:36707 (local address 127.21.51.131:51623)
0809 16:23:13.451703 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0809 16:23:13.451768 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.51.190:36707: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":86,"threads_started":1}
I20260809 16:23:14.289515 13135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:23:14.289242 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:36707 (local address 127.21.51.129:59379)
0809 16:23:14.289408 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0809 16:23:14.289452 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.51.190:36707: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":78,"threads_started":1}
I20260809 16:23:14.330675 13136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:23:14.330457 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:36707 (local address 127.21.51.130:54365)
0809 16:23:14.330599 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0809 16:23:14.330635 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.51.190:36707: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":84,"threads_started":1}
I20260809 16:23:14.452497 13137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:23:14.452267 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:36707 (local address 127.21.51.131:33279)
0809 16:23:14.452419 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0809 16:23:14.452455 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.51.190:36707: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":88,"threads_started":1}
I20260809 16:23:15.290223 13139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:23:15.289996 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:36707 (local address 127.21.51.129:60273)
0809 16:23:15.290140 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0809 16:23:15.290177 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.51.190:36707: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":86,"threads_started":1}
I20260809 16:23:15.331383 13141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:23:15.331102 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:36707 (local address 127.21.51.130:46957)
0809 16:23:15.331304 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0809 16:23:15.331342 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.51.190:36707: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":61,"threads_started":1}
I20260809 16:23:15.453159 13142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:23:15.452959 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:36707 (local address 127.21.51.131:60449)
0809 16:23:15.453079 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0809 16:23:15.453117 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.51.190:36707: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":59,"threads_started":1}
I20260809 16:23:16.290951 13144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:23:16.290723 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:36707 (local address 127.21.51.129:54013)
0809 16:23:16.290865 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0809 16:23:16.290903 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.51.190:36707: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":68,"threads_started":1}
I20260809 16:23:16.332170 13146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:23:16.331949 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:36707 (local address 127.21.51.130:45531)
0809 16:23:16.332091 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0809 16:23:16.332127 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.51.190:36707: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":65,"threads_started":1}
I20260809 16:23:16.440657 13147 raft_consensus.cc:482] T 8e7536df343f4e51853cff79a145f428 P 57576785ec93441f9b827f3a8b6df2ed [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 16:23:16.440864 13147 raft_consensus.cc:504] T 8e7536df343f4e51853cff79a145f428 P 57576785ec93441f9b827f3a8b6df2ed [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "274512f1889a4254b485fef854967321" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 42799 } } peers { permanent_uuid: "5ba1dc75657f4418b6903fa6ceed3d8b" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 46397 } } peers { permanent_uuid: "57576785ec93441f9b827f3a8b6df2ed" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 41875 } }
I20260809 16:23:16.441092 13147 leader_election.cc:288] T 8e7536df343f4e51853cff79a145f428 P 57576785ec93441f9b827f3a8b6df2ed [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 274512f1889a4254b485fef854967321 (127.21.51.131:42799), 5ba1dc75657f4418b6903fa6ceed3d8b (127.21.51.130:46397)
W20260809 16:23:16.441533 12492 leader_election.cc:341] T 8e7536df343f4e51853cff79a145f428 P 57576785ec93441f9b827f3a8b6df2ed [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 5ba1dc75657f4418b6903fa6ceed3d8b (127.21.51.130:46397): Not found: Tablet not found: 8e7536df343f4e51853cff79a145f428
W20260809 16:23:16.441630 12485 leader_election.cc:341] T 8e7536df343f4e51853cff79a145f428 P 57576785ec93441f9b827f3a8b6df2ed [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 274512f1889a4254b485fef854967321 (127.21.51.131:42799): Not found: Tablet not found: 8e7536df343f4e51853cff79a145f428
I20260809 16:23:16.441692 12485 leader_election.cc:302] T 8e7536df343f4e51853cff79a145f428 P 57576785ec93441f9b827f3a8b6df2ed [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57576785ec93441f9b827f3a8b6df2ed; no voters: 274512f1889a4254b485fef854967321, 5ba1dc75657f4418b6903fa6ceed3d8b
I20260809 16:23:16.441897 13147 raft_consensus.cc:2726] T 8e7536df343f4e51853cff79a145f428 P 57576785ec93441f9b827f3a8b6df2ed [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 16:23:16.453869 13148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:23:16.453674 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:36707 (local address 127.21.51.131:44995)
0809 16:23:16.453794 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0809 16:23:16.453827 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.51.190:36707: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":67,"threads_started":1}
I20260809 16:23:17.291819 13149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:23:17.291535 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:36707 (local address 127.21.51.129:51731)
0809 16:23:17.291725 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0809 16:23:17.291767 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.51.190:36707: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":67,"threads_started":1}
I20260809 16:23:17.333069 13152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:23:17.332874 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:36707 (local address 127.21.51.130:55487)
0809 16:23:17.332993 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0809 16:23:17.333026 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.51.190:36707: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":63,"threads_started":1}
I20260809 16:23:17.449928 21710 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrNqUCn/build/release/bin/kudu with pid 12419
I20260809 16:23:17.457053 13153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:23:17.454357 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:36707 (local address 127.21.51.131:43705)
0809 16:23:17.456944 (+ 2587us) negotiation.cc:107] Waiting for socket to connect
0809 16:23:17.456990 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.51.190:36707: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2566,"thread_start_us":1191,"threads_started":1}
I20260809 16:23:17.463528 21710 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrNqUCn/build/release/bin/kudu with pid 12675
I20260809 16:23:17.476608 21710 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrNqUCn/build/release/bin/kudu with pid 12879
2026-08-09T16:23:17Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (18577 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292417722232-21710-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 16:23:17 dist-test-slave-s11t krb5kdc[13156](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 16:23:17 dist-test-slave-s11t krb5kdc[13156](info): set up 2 sockets
Aug 09 16:23:17 dist-test-slave-s11t krb5kdc[13156](info): commencing operation
krb5kdc: starting...
W20260809 16:23:18.762951 21710 mini_kdc.cc:121] Time spent starting KDC: real 1.258s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292417722232-21710-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292417722232-21710-0/krb5kdc/test-user.keytab.
Aug 09 16:23:18 dist-test-slave-s11t krb5kdc[13156](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292598, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T16:23:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T16:23:18Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.51.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.51.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.51.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.190.keytab.
Entry for principal kudu/127.21.51.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.190.keytab.
I20260809 16:23:18.891670 21710 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrNqUCn/build/release/bin/kudu
/tmp/dist-test-taskrNqUCn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292417722232-21710-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292417722232-21710-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292417722232-21710-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292417722232-21710-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.51.190:44381
--webserver_interface=127.21.51.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.190.keytab
--principal=kudu/127.21.51.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.51.148:34103
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.51.190:44381
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292417722232-21710-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292417722232-21710-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292417722232-21710-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:23:18.975095 13174 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:23:18.975340 13174 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:23:18.975379 13174 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:23:18.976521 13174 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260809 16:23:18.976591 13174 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260809 16:23:18.976614 13174 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260809 16:23:18.976645 13174 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 16:23:18.976673 13174 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:23:18.976722 13174 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:23:18.976747 13174 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 16:23:18.976768 13174 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 16:23:18.978358 13174 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.51.148:34103
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292417722232-21710-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292417722232-21710-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.21.51.190:44381
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.190.keytab
--principal=kudu/127.21.51.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.51.190:44381
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292417722232-21710-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.51.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13174
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292417722232-21710-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:23:18.978641 13174 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:23:18.978885 13174 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:23:18.984062 13184 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:23:18.984081 13179 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:23:18.984112 13180 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:23:18.984097 13174 server_base.cc:1034] running on GCE node
I20260809 16:23:18.984552 13174 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:23:18.984899 13174 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:23:18.986052 13174 hybrid_clock.cc:648] HybridClock initialized: now 1786292598986027 us; error 38 us; skew 500 ppm
Aug 09 16:23:18 dist-test-slave-s11t krb5kdc[13156](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292598, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.51.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:23:18.988505 13174 init.cc:377] Logged in from keytab as kudu/127.21.51.190@KRBTEST.COM (short username kudu)
I20260809 16:23:18.989045 13174 webserver.cc:460] Webserver started at http://127.21.51.190:34771/ using document root <none> and password file <none>
I20260809 16:23:18.989305 13174 fs_manager.cc:359] Metadata directory not provided
I20260809 16:23:18.989382 13174 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:23:18.989540 13174 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:23:18.990573 13174 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292417722232-21710-0/minicluster-data/master-0/data/instance:
uuid: "8c44acc0f3e8446693dc64d77163f6b4"
format_stamp: "Formatted at 2026-08-09 16:23:18 on dist-test-slave-s11t"
server_key: "feee4944f16d07b04e52e984838a350c"
server_key_iv: "b5fdc51511f5019e4a395d2478e913f1"
server_key_version: "encryptionkey@0"
I20260809 16:23:18.991026 13174 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292417722232-21710-0/minicluster-data/master-0/wal/instance:
uuid: "8c44acc0f3e8446693dc64d77163f6b4"
format_stamp: "Formatted at 2026-08-09 16:23:18 on dist-test-slave-s11t"
server_key: "feee4944f16d07b04e52e984838a350c"
server_key_iv: "b5fdc51511f5019e4a395d2478e913f1"
server_key_version: "encryptionkey@0"
I20260809 16:23:18.992404 13174 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 16:23:18.993261 13194 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:23:18.993482 13174 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260809 16:23:18.993597 13174 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292417722232-21710-0/minicluster-data/master-0/data,/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292417722232-21710-0/minicluster-data/master-0/wal
uuid: "8c44acc0f3e8446693dc64d77163f6b4"
format_stamp: "Formatted at 2026-08-09 16:23:18 on dist-test-slave-s11t"
server_key: "feee4944f16d07b04e52e984838a350c"
server_key_iv: "b5fdc51511f5019e4a395d2478e913f1"
server_key_version: "encryptionkey@0"
I20260809 16:23:18.993685 13174 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292417722232-21710-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292417722232-21710-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292417722232-21710-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:23:19.006217 13174 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:23:19.008191 13174 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:23:19.008376 13174 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:23:19.012804 13174 rpc_server.cc:307] RPC server started. Bound to: 127.21.51.190:44381
I20260809 16:23:19.012840 13274 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.51.190:44381 every 8 connection(s)
I20260809 16:23:19.013204 13174 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292417722232-21710-0/minicluster-data/master-0/data/info.pb
I20260809 16:23:19.014101 13275 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:23:19.016399 13275 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8c44acc0f3e8446693dc64d77163f6b4: Bootstrap starting.
I20260809 16:23:19.016556 21710 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrNqUCn/build/release/bin/kudu as pid 13174
I20260809 16:23:19.016652 21710 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292417722232-21710-0/minicluster-data/master-0/wal/instance
I20260809 16:23:19.016934 21710 external_mini_cluster.cc:1428] Setting key d4c4636edb472d9a6478c3aea9a01f26
I20260809 16:23:19.017218 13275 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8c44acc0f3e8446693dc64d77163f6b4: Neither blocks nor log segments found. Creating new log.
I20260809 16:23:19.017593 13275 log.cc:824] T 00000000000000000000000000000000 P 8c44acc0f3e8446693dc64d77163f6b4: Log is configured to *not* fsync() on all Append() calls
I20260809 16:23:19.018337 13275 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8c44acc0f3e8446693dc64d77163f6b4: No bootstrap required, opened a new log
I20260809 16:23:19.020350 13275 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8c44acc0f3e8446693dc64d77163f6b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c44acc0f3e8446693dc64d77163f6b4" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 44381 } }
I20260809 16:23:19.020526 13275 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8c44acc0f3e8446693dc64d77163f6b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:23:19.020579 13275 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8c44acc0f3e8446693dc64d77163f6b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c44acc0f3e8446693dc64d77163f6b4, State: Initialized, Role: FOLLOWER
I20260809 16:23:19.020787 13275 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8c44acc0f3e8446693dc64d77163f6b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c44acc0f3e8446693dc64d77163f6b4" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 44381 } }
I20260809 16:23:19.020895 13275 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8c44acc0f3e8446693dc64d77163f6b4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 16:23:19.020941 13275 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8c44acc0f3e8446693dc64d77163f6b4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 16:23:19.020992 13275 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8c44acc0f3e8446693dc64d77163f6b4 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:23:19.021561 13275 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8c44acc0f3e8446693dc64d77163f6b4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c44acc0f3e8446693dc64d77163f6b4" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 44381 } }
I20260809 16:23:19.021706 13275 leader_election.cc:302] T 00000000000000000000000000000000 P 8c44acc0f3e8446693dc64d77163f6b4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8c44acc0f3e8446693dc64d77163f6b4; no voters:
I20260809 16:23:19.021914 13275 leader_election.cc:288] T 00000000000000000000000000000000 P 8c44acc0f3e8446693dc64d77163f6b4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 16:23:19.021946 13282 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8c44acc0f3e8446693dc64d77163f6b4 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:23:19.022063 13282 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8c44acc0f3e8446693dc64d77163f6b4 [term 1 LEADER]: Becoming Leader. State: Replica: 8c44acc0f3e8446693dc64d77163f6b4, State: Running, Role: LEADER
I20260809 16:23:19.022224 13275 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8c44acc0f3e8446693dc64d77163f6b4 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 16:23:19.022226 13282 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8c44acc0f3e8446693dc64d77163f6b4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c44acc0f3e8446693dc64d77163f6b4" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 44381 } }
I20260809 16:23:19.022677 13282 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8c44acc0f3e8446693dc64d77163f6b4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8c44acc0f3e8446693dc64d77163f6b4. Latest consensus state: current_term: 1 leader_uuid: "8c44acc0f3e8446693dc64d77163f6b4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c44acc0f3e8446693dc64d77163f6b4" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 44381 } } }
I20260809 16:23:19.022713 13283 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8c44acc0f3e8446693dc64d77163f6b4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8c44acc0f3e8446693dc64d77163f6b4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c44acc0f3e8446693dc64d77163f6b4" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 44381 } } }
I20260809 16:23:19.022841 13282 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8c44acc0f3e8446693dc64d77163f6b4 [sys.catalog]: This master's current role is: LEADER
W20260809 16:23:19.024291 13305 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 8c44acc0f3e8446693dc64d77163f6b4: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 16:23:19.024339 13305 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 16:23:19.024556 13312 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 16:23:19.024650 13283 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8c44acc0f3e8446693dc64d77163f6b4 [sys.catalog]: This master's current role is: LEADER
I20260809 16:23:19.024931 13312 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 09 16:23:19 dist-test-slave-s11t krb5kdc[13156](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292598, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.51.190@KRBTEST.COM
I20260809 16:23:19.026780 13312 catalog_manager.cc:1347] Generated new cluster ID: ae92579412e5493faccfdf6bb3f0cb1f
I20260809 16:23:19.026846 13312 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 16:23:19.029906 13279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:23:19.017449 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41480 (local address 127.21.51.190:44381)
0809 16:23:19.017662 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0809 16:23:19.017673 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 16:23:19.017707 (+ 34us) server_negotiation.cc:407] Connection header received
0809 16:23:19.018253 (+ 546us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:23:19.018260 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:23:19.018316 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:23:19.018480 (+ 164us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:23:19.019387 (+ 907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:23:19.020382 (+ 995us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:23:19.021528 (+ 1146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:23:19.022185 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:23:19.022334 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:23:19.025650 (+ 3316us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:23:19.025653 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:23:19.025654 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:23:19.025667 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:23:19.027955 (+ 2288us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:23:19.028368 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:23:19.028371 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:23:19.028371 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:23:19.028470 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:23:19.028659 (+ 189us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:23:19.028660 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:23:19.028660 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:23:19.028980 (+ 320us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:23:19.029127 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0809 16:23:19.029561 (+ 434us) server_negotiation.cc:299] Negotiation successful
0809 16:23:19.029699 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":158,"thread_start_us":85,"threads_started":1}
I20260809 16:23:19.055109 13312 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 16:23:19.055758 13312 catalog_manager.cc:1504] Loading token signing keys...
I20260809 16:23:19.058261 13312 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8c44acc0f3e8446693dc64d77163f6b4: Generated new TSK 0
I20260809 16:23:19.058498 13312 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.21.51.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.51.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.51.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.129.keytab.
Entry for principal kudu/127.21.51.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.129.keytab.
I20260809 16:23:19.092746 21710 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrNqUCn/build/release/bin/kudu
/tmp/dist-test-taskrNqUCn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292417722232-21710-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292417722232-21710-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292417722232-21710-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292417722232-21710-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.51.129:0
--local_ip_for_outbound_sockets=127.21.51.129
--webserver_interface=127.21.51.129
--webserver_port=0
--tserver_master_addrs=127.21.51.190:44381
--keytab_file=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.129.keytab
--principal=kudu/127.21.51.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.51.148:34103
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292417722232-21710-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292417722232-21710-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292417722232-21710-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:23:19.175840 13316 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:23:19.176043 13316 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:23:19.176065 13316 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:23:19.177222 13316 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:23:19.177273 13316 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:23:19.177299 13316 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.51.129
I20260809 16:23:19.178836 13316 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.51.148:34103
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292417722232-21710-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292417722232-21710-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.129.keytab
--principal=kudu/127.21.51.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.51.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292417722232-21710-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.51.129
--webserver_port=0
--tserver_master_addrs=127.21.51.190:44381
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13316
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.51.129
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292417722232-21710-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:23:19.179031 13316 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:23:19.179258 13316 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 16:23:19.184645 13316 server_base.cc:1034] running on GCE node
W20260809 16:23:19.184625 13330 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:23:19.184612 13326 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:23:19.184625 13325 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:23:19.185146 13316 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:23:19.185398 13316 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:23:19.186543 13316 hybrid_clock.cc:648] HybridClock initialized: now 1786292599186520 us; error 45 us; skew 500 ppm
Aug 09 16:23:19 dist-test-slave-s11t krb5kdc[13156](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292599, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.51.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:23:19.189353 13316 init.cc:377] Logged in from keytab as kudu/127.21.51.129@KRBTEST.COM (short username kudu)
I20260809 16:23:19.189793 13316 webserver.cc:460] Webserver started at http://127.21.51.129:40013/ using document root <none> and password file <none>
I20260809 16:23:19.190024 13316 fs_manager.cc:359] Metadata directory not provided
I20260809 16:23:19.190073 13316 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:23:19.190208 13316 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:23:19.191244 13316 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292417722232-21710-0/minicluster-data/ts-0/data/instance:
uuid: "7d67bdf64e544c339ca52e32fcd4a5a7"
format_stamp: "Formatted at 2026-08-09 16:23:19 on dist-test-slave-s11t"
server_key: "3899e01c0413848a2deae05fbcc2cec1"
server_key_iv: "6f19b55e88ae041ff9a866bc626841a2"
server_key_version: "encryptionkey@0"
I20260809 16:23:19.191620 13316 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292417722232-21710-0/minicluster-data/ts-0/wal/instance:
uuid: "7d67bdf64e544c339ca52e32fcd4a5a7"
format_stamp: "Formatted at 2026-08-09 16:23:19 on dist-test-slave-s11t"
server_key: "3899e01c0413848a2deae05fbcc2cec1"
server_key_iv: "6f19b55e88ae041ff9a866bc626841a2"
server_key_version: "encryptionkey@0"
I20260809 16:23:19.193074 13316 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 16:23:19.194023 13339 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:23:19.194265 13316 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 16:23:19.194375 13316 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292417722232-21710-0/minicluster-data/ts-0/data,/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292417722232-21710-0/minicluster-data/ts-0/wal
uuid: "7d67bdf64e544c339ca52e32fcd4a5a7"
format_stamp: "Formatted at 2026-08-09 16:23:19 on dist-test-slave-s11t"
server_key: "3899e01c0413848a2deae05fbcc2cec1"
server_key_iv: "6f19b55e88ae041ff9a866bc626841a2"
server_key_version: "encryptionkey@0"
I20260809 16:23:19.194464 13316 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292417722232-21710-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292417722232-21710-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292417722232-21710-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:23:19.213739 13316 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:23:19.216113 13316 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:23:19.216293 13316 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:23:19.216552 13316 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:23:19.216964 13316 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:23:19.217026 13316 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:23:19.217070 13316 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:23:19.217099 13316 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:23:19.223542 13316 rpc_server.cc:307] RPC server started. Bound to: 127.21.51.129:37335
I20260809 16:23:19.223618 13499 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.51.129:37335 every 8 connection(s)
I20260809 16:23:19.224184 13316 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292417722232-21710-0/minicluster-data/ts-0/data/info.pb
I20260809 16:23:19.228361 21710 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrNqUCn/build/release/bin/kudu as pid 13316
I20260809 16:23:19.228492 21710 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292417722232-21710-0/minicluster-data/ts-0/wal/instance
I20260809 16:23:19.228794 21710 external_mini_cluster.cc:1428] Setting key 12b3ca362e39aea007c0ca7596e8e4eb
Aug 09 16:23:19 dist-test-slave-s11t krb5kdc[13156](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292599, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.51.129@KRBTEST.COM for kudu/127.21.51.190@KRBTEST.COM
I20260809 16:23:19.234737 13279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:23:19.225185 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.51.129:34437 (local address 127.21.51.190:44381)
0809 16:23:19.225262 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0809 16:23:19.225265 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 16:23:19.225462 (+ 197us) server_negotiation.cc:407] Connection header received
0809 16:23:19.226000 (+ 538us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:23:19.226001 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:23:19.226014 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:23:19.226071 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:23:19.227061 (+ 990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:23:19.227585 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:23:19.228655 (+ 1070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:23:19.228976 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:23:19.229077 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:23:19.231477 (+ 2400us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:23:19.231478 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:23:19.231478 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:23:19.231486 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:23:19.233260 (+ 1774us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:23:19.233682 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:23:19.233683 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:23:19.233683 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:23:19.233742 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:23:19.234024 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:23:19.234025 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:23:19.234026 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:23:19.234149 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:23:19.234247 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0809 16:23:19.234556 (+ 309us) server_negotiation.cc:299] Negotiation successful
0809 16:23:19.234650 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260809 16:23:19.234929 13502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:23:19.225109 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:44381 (local address 127.21.51.129:34437)
0809 16:23:19.225317 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0809 16:23:19.225369 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0809 16:23:19.225878 (+ 509us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:23:19.226142 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:23:19.226147 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:23:19.226742 (+ 595us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:23:19.226930 (+ 188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:23:19.226935 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:23:19.227696 (+ 761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:23:19.227697 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:23:19.228555 (+ 858us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:23:19.228558 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:23:19.229089 (+ 531us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:23:19.229090 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:23:19.229255 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:23:19.229690 (+ 435us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:23:19.229697 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:23:19.231369 (+ 1672us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:23:19.233353 (+ 1984us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:23:19.233359 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:23:19.233362 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:23:19.233592 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:23:19.233832 (+ 240us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:23:19.233833 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:23:19.233833 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:23:19.233949 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:23:19.234240 (+ 291us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:23:19.234247 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:23:19.234388 (+ 141us) client_negotiation.cc:769] Sending connection context
0809 16:23:19.234479 (+ 91us) client_negotiation.cc:240] Negotiation successful
0809 16:23:19.234670 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":111,"threads_started":1}
I20260809 16:23:19.235529 13500 heartbeater.cc:347] Connected to a master server at 127.21.51.190:44381
I20260809 16:23:19.235623 13500 heartbeater.cc:464] Registering TS with master...
I20260809 16:23:19.235829 13500 heartbeater.cc:511] Master 127.21.51.190:44381 requested a full tablet report, sending...
I20260809 16:23:19.236392 13221 ts_manager.cc:194] Registered new tserver with Master: 7d67bdf64e544c339ca52e32fcd4a5a7 (127.21.51.129:37335)
I20260809 16:23:19.237097 13221 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.51.129@KRBTEST.COM'} at 127.21.51.129:34437
WARNING: no policy specified for kudu/127.21.51.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.51.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.51.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.130.keytab.
Entry for principal kudu/127.21.51.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.130.keytab.
I20260809 16:23:19.255940 21710 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrNqUCn/build/release/bin/kudu
/tmp/dist-test-taskrNqUCn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292417722232-21710-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292417722232-21710-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292417722232-21710-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292417722232-21710-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.51.130:0
--local_ip_for_outbound_sockets=127.21.51.130
--webserver_interface=127.21.51.130
--webserver_port=0
--tserver_master_addrs=127.21.51.190:44381
--keytab_file=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.130.keytab
--principal=kudu/127.21.51.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.51.148:34103
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292417722232-21710-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292417722232-21710-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292417722232-21710-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:23:19.338248 13510 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:23:19.338445 13510 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:23:19.338467 13510 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:23:19.339602 13510 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:23:19.339644 13510 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:23:19.339670 13510 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.51.130
I20260809 16:23:19.341204 13510 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.51.148:34103
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292417722232-21710-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292417722232-21710-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.130.keytab
--principal=kudu/127.21.51.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.51.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292417722232-21710-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.51.130
--webserver_port=0
--tserver_master_addrs=127.21.51.190:44381
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13510
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.51.130
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292417722232-21710-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:23:19.341449 13510 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:23:19.341692 13510 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:23:19.346810 13518 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:23:19.346820 13520 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:23:19.346820 13516 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:23:19.347052 13510 server_base.cc:1034] running on GCE node
I20260809 16:23:19.347246 13510 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:23:19.347496 13510 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:23:19.348675 13510 hybrid_clock.cc:648] HybridClock initialized: now 1786292599348678 us; error 49 us; skew 500 ppm
Aug 09 16:23:19 dist-test-slave-s11t krb5kdc[13156](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292599, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.51.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:23:19.351277 13510 init.cc:377] Logged in from keytab as kudu/127.21.51.130@KRBTEST.COM (short username kudu)
I20260809 16:23:19.351764 13510 webserver.cc:460] Webserver started at http://127.21.51.130:36157/ using document root <none> and password file <none>
I20260809 16:23:19.352017 13510 fs_manager.cc:359] Metadata directory not provided
I20260809 16:23:19.352092 13510 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:23:19.352247 13510 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:23:19.353343 13510 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292417722232-21710-0/minicluster-data/ts-1/data/instance:
uuid: "108d298be817495d987fd45a969d96ba"
format_stamp: "Formatted at 2026-08-09 16:23:19 on dist-test-slave-s11t"
server_key: "61136fc1c27ac7e207ba3360818e7bf6"
server_key_iv: "344bc88d0dd8954136aaa8398445bb69"
server_key_version: "encryptionkey@0"
I20260809 16:23:19.353722 13510 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292417722232-21710-0/minicluster-data/ts-1/wal/instance:
uuid: "108d298be817495d987fd45a969d96ba"
format_stamp: "Formatted at 2026-08-09 16:23:19 on dist-test-slave-s11t"
server_key: "61136fc1c27ac7e207ba3360818e7bf6"
server_key_iv: "344bc88d0dd8954136aaa8398445bb69"
server_key_version: "encryptionkey@0"
I20260809 16:23:19.355062 13510 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 16:23:19.355795 13532 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:23:19.355983 13510 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 16:23:19.356077 13510 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292417722232-21710-0/minicluster-data/ts-1/data,/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292417722232-21710-0/minicluster-data/ts-1/wal
uuid: "108d298be817495d987fd45a969d96ba"
format_stamp: "Formatted at 2026-08-09 16:23:19 on dist-test-slave-s11t"
server_key: "61136fc1c27ac7e207ba3360818e7bf6"
server_key_iv: "344bc88d0dd8954136aaa8398445bb69"
server_key_version: "encryptionkey@0"
I20260809 16:23:19.356163 13510 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292417722232-21710-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292417722232-21710-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292417722232-21710-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:23:19.365336 13510 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:23:19.367647 13510 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:23:19.367818 13510 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:23:19.368055 13510 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:23:19.368364 13510 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:23:19.368430 13510 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:23:19.368464 13510 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:23:19.368491 13510 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:23:19.374764 13510 rpc_server.cc:307] RPC server started. Bound to: 127.21.51.130:43191
I20260809 16:23:19.374800 13700 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.51.130:43191 every 8 connection(s)
I20260809 16:23:19.375128 13510 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292417722232-21710-0/minicluster-data/ts-1/data/info.pb
Aug 09 16:23:19 dist-test-slave-s11t krb5kdc[13156](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292599, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.51.130@KRBTEST.COM for kudu/127.21.51.190@KRBTEST.COM
I20260809 16:23:19.381352 21710 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrNqUCn/build/release/bin/kudu as pid 13510
I20260809 16:23:19.381480 21710 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292417722232-21710-0/minicluster-data/ts-1/wal/instance
I20260809 16:23:19.381702 21710 external_mini_cluster.cc:1428] Setting key 4b3945ebe850edc82d90194aaba451dc
I20260809 16:23:19.384644 13279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:23:19.375912 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.51.130:47755 (local address 127.21.51.190:44381)
0809 16:23:19.376002 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0809 16:23:19.376004 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:23:19.376167 (+ 163us) server_negotiation.cc:407] Connection header received
0809 16:23:19.376617 (+ 450us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:23:19.376618 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:23:19.376628 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:23:19.376729 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:23:19.377621 (+ 892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:23:19.378117 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:23:19.379166 (+ 1049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:23:19.379369 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:23:19.379429 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:23:19.381579 (+ 2150us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:23:19.381581 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:23:19.381581 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:23:19.381589 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:23:19.383265 (+ 1676us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:23:19.383663 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:23:19.383665 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:23:19.383665 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:23:19.383732 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:23:19.383967 (+ 235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:23:19.383968 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:23:19.383969 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:23:19.384117 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:23:19.384188 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0809 16:23:19.384470 (+ 282us) server_negotiation.cc:299] Negotiation successful
0809 16:23:19.384557 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260809 16:23:19.384881 13704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:23:19.375855 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:44381 (local address 127.21.51.130:47755)
0809 16:23:19.376047 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0809 16:23:19.376080 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0809 16:23:19.376533 (+ 453us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:23:19.376792 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:23:19.376796 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:23:19.377346 (+ 550us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:23:19.377539 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:23:19.377543 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:23:19.378201 (+ 658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:23:19.378201 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:23:19.379059 (+ 858us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:23:19.379061 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:23:19.379453 (+ 392us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:23:19.379454 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:23:19.379553 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:23:19.380055 (+ 502us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:23:19.380063 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:23:19.381474 (+ 1411us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:23:19.383349 (+ 1875us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:23:19.383355 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:23:19.383357 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:23:19.383580 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:23:19.383803 (+ 223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:23:19.383803 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:23:19.383804 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:23:19.383897 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:23:19.384181 (+ 284us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:23:19.384185 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:23:19.384290 (+ 105us) client_negotiation.cc:769] Sending connection context
0809 16:23:19.384420 (+ 130us) client_negotiation.cc:240] Negotiation successful
0809 16:23:19.384576 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":110,"threads_started":1}
I20260809 16:23:19.385311 13701 heartbeater.cc:347] Connected to a master server at 127.21.51.190:44381
I20260809 16:23:19.385406 13701 heartbeater.cc:464] Registering TS with master...
I20260809 16:23:19.385541 13701 heartbeater.cc:511] Master 127.21.51.190:44381 requested a full tablet report, sending...
I20260809 16:23:19.385880 13221 ts_manager.cc:194] Registered new tserver with Master: 108d298be817495d987fd45a969d96ba (127.21.51.130:43191)
I20260809 16:23:19.386270 13221 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.51.130@KRBTEST.COM'} at 127.21.51.130:47755
WARNING: no policy specified for kudu/127.21.51.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.51.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.51.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.131.keytab.
Entry for principal kudu/127.21.51.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.131.keytab.
I20260809 16:23:19.407673 21710 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrNqUCn/build/release/bin/kudu
/tmp/dist-test-taskrNqUCn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292417722232-21710-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292417722232-21710-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292417722232-21710-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292417722232-21710-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.51.131:0
--local_ip_for_outbound_sockets=127.21.51.131
--webserver_interface=127.21.51.131
--webserver_port=0
--tserver_master_addrs=127.21.51.190:44381
--keytab_file=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.131.keytab
--principal=kudu/127.21.51.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.51.148:34103
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292417722232-21710-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292417722232-21710-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292417722232-21710-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:23:19.488981 13712 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:23:19.489176 13712 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:23:19.489197 13712 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:23:19.490327 13712 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:23:19.490372 13712 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:23:19.490398 13712 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.51.131
I20260809 16:23:19.491802 13712 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.51.148:34103
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292417722232-21710-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292417722232-21710-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.131.keytab
--principal=kudu/127.21.51.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.51.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292417722232-21710-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.51.131
--webserver_port=0
--tserver_master_addrs=127.21.51.190:44381
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13712
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.51.131
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292417722232-21710-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:23:19.492092 13712 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:23:19.492337 13712 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:23:19.497581 13719 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:23:19.497581 13724 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:23:19.497615 13721 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:23:19.497804 13712 server_base.cc:1034] running on GCE node
I20260809 16:23:19.498031 13712 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:23:19.498342 13712 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:23:19.499495 13712 hybrid_clock.cc:648] HybridClock initialized: now 1786292599499487 us; error 30 us; skew 500 ppm
Aug 09 16:23:19 dist-test-slave-s11t krb5kdc[13156](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292599, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.51.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:23:19.502147 13712 init.cc:377] Logged in from keytab as kudu/127.21.51.131@KRBTEST.COM (short username kudu)
I20260809 16:23:19.502528 13712 webserver.cc:460] Webserver started at http://127.21.51.131:37545/ using document root <none> and password file <none>
I20260809 16:23:19.502722 13712 fs_manager.cc:359] Metadata directory not provided
I20260809 16:23:19.502763 13712 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:23:19.502853 13712 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:23:19.503799 13712 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292417722232-21710-0/minicluster-data/ts-2/data/instance:
uuid: "6aa3f90a2d2042e08594f1dd05d7fe30"
format_stamp: "Formatted at 2026-08-09 16:23:19 on dist-test-slave-s11t"
server_key: "3002adbb5d94446f03cb87eaed035f30"
server_key_iv: "e451942046f83051fdc9d91823d2a973"
server_key_version: "encryptionkey@0"
I20260809 16:23:19.504104 13712 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292417722232-21710-0/minicluster-data/ts-2/wal/instance:
uuid: "6aa3f90a2d2042e08594f1dd05d7fe30"
format_stamp: "Formatted at 2026-08-09 16:23:19 on dist-test-slave-s11t"
server_key: "3002adbb5d94446f03cb87eaed035f30"
server_key_iv: "e451942046f83051fdc9d91823d2a973"
server_key_version: "encryptionkey@0"
I20260809 16:23:19.505505 13712 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 16:23:19.506290 13736 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:23:19.506516 13712 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:23:19.506603 13712 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292417722232-21710-0/minicluster-data/ts-2/data,/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292417722232-21710-0/minicluster-data/ts-2/wal
uuid: "6aa3f90a2d2042e08594f1dd05d7fe30"
format_stamp: "Formatted at 2026-08-09 16:23:19 on dist-test-slave-s11t"
server_key: "3002adbb5d94446f03cb87eaed035f30"
server_key_iv: "e451942046f83051fdc9d91823d2a973"
server_key_version: "encryptionkey@0"
I20260809 16:23:19.506655 13712 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292417722232-21710-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292417722232-21710-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292417722232-21710-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:23:19.515316 13712 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:23:19.517613 13712 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:23:19.517724 13712 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:23:19.517892 13712 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:23:19.518128 13712 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:23:19.518157 13712 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:23:19.518178 13712 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:23:19.518261 13712 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:23:19.524461 13712 rpc_server.cc:307] RPC server started. Bound to: 127.21.51.131:40517
I20260809 16:23:19.524534 13909 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.51.131:40517 every 8 connection(s)
I20260809 16:23:19.524865 13712 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292417722232-21710-0/minicluster-data/ts-2/data/info.pb
Aug 09 16:23:19 dist-test-slave-s11t krb5kdc[13156](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292599, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.51.131@KRBTEST.COM for kudu/127.21.51.190@KRBTEST.COM
I20260809 16:23:19.532524 21710 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrNqUCn/build/release/bin/kudu as pid 13712
I20260809 16:23:19.532644 21710 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292417722232-21710-0/minicluster-data/ts-2/wal/instance
I20260809 16:23:19.532883 21710 external_mini_cluster.cc:1428] Setting key 1a28879177be6e4529e1adc0c729751a
I20260809 16:23:19.534420 13279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:23:19.525632 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.51.131:42781 (local address 127.21.51.190:44381)
0809 16:23:19.525676 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0809 16:23:19.525677 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 16:23:19.525900 (+ 223us) server_negotiation.cc:407] Connection header received
0809 16:23:19.526437 (+ 537us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:23:19.526438 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:23:19.526451 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:23:19.526514 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:23:19.527439 (+ 925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:23:19.527962 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:23:19.529048 (+ 1086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:23:19.529265 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:23:19.529352 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:23:19.531529 (+ 2177us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:23:19.531531 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:23:19.531531 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:23:19.531539 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:23:19.532847 (+ 1308us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:23:19.533384 (+ 537us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:23:19.533385 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:23:19.533385 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:23:19.533479 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:23:19.533826 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:23:19.533827 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:23:19.533827 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:23:19.533981 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:23:19.534043 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0809 16:23:19.534286 (+ 243us) server_negotiation.cc:299] Negotiation successful
0809 16:23:19.534356 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260809 16:23:19.534572 13918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:23:19.525594 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:44381 (local address 127.21.51.131:42781)
0809 16:23:19.525777 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0809 16:23:19.525812 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0809 16:23:19.526302 (+ 490us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:23:19.526581 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:23:19.526585 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:23:19.527152 (+ 567us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:23:19.527333 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:23:19.527337 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:23:19.528074 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:23:19.528075 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:23:19.528956 (+ 881us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:23:19.528959 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:23:19.529356 (+ 397us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:23:19.529357 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:23:19.529507 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:23:19.530039 (+ 532us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:23:19.530047 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:23:19.531427 (+ 1380us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:23:19.532969 (+ 1542us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:23:19.532976 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:23:19.532978 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:23:19.533282 (+ 304us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:23:19.533552 (+ 270us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:23:19.533553 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:23:19.533554 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:23:19.533696 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:23:19.534038 (+ 342us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:23:19.534044 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:23:19.534166 (+ 122us) client_negotiation.cc:769] Sending connection context
0809 16:23:19.534252 (+ 86us) client_negotiation.cc:240] Negotiation successful
0809 16:23:19.534370 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":89,"threads_started":1}
I20260809 16:23:19.534994 13910 heartbeater.cc:347] Connected to a master server at 127.21.51.190:44381
I20260809 16:23:19.535105 13910 heartbeater.cc:464] Registering TS with master...
I20260809 16:23:19.535291 13910 heartbeater.cc:511] Master 127.21.51.190:44381 requested a full tablet report, sending...
I20260809 16:23:19.535658 13221 ts_manager.cc:194] Registered new tserver with Master: 6aa3f90a2d2042e08594f1dd05d7fe30 (127.21.51.131:40517)
I20260809 16:23:19.536020 13221 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.51.131@KRBTEST.COM'} at 127.21.51.131:42781
I20260809 16:23:19.545281 21710 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 16:23:19.552623 13279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:23:19.546596 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41496 (local address 127.21.51.190:44381)
0809 16:23:19.546668 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0809 16:23:19.546670 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:23:19.546778 (+ 108us) server_negotiation.cc:407] Connection header received
0809 16:23:19.546803 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:23:19.546804 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:23:19.546813 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:23:19.546867 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:23:19.547413 (+ 546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:23:19.547910 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:23:19.548617 (+ 707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:23:19.548906 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:23:19.548964 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:23:19.550074 (+ 1110us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:23:19.550076 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:23:19.550076 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:23:19.550084 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:23:19.551519 (+ 1435us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:23:19.551817 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:23:19.551818 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:23:19.551818 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:23:19.551870 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:23:19.552087 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:23:19.552089 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:23:19.552089 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:23:19.552261 (+ 172us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:23:19.552352 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0809 16:23:19.552444 (+ 92us) server_negotiation.cc:299] Negotiation successful
0809 16:23:19.552510 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260809 16:23:19.554329 13221 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41496:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 16:23:19.555925 13221 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 16:23:19.562678 13929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:23:19.558238 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.131:40517 (local address 127.0.0.1:48312)
0809 16:23:19.558617 (+ 379us) negotiation.cc:107] Waiting for socket to connect
0809 16:23:19.558628 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 16:23:19.558713 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:23:19.559086 (+ 373us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:23:19.559089 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:23:19.559091 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:23:19.559166 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:23:19.559168 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:23:19.560329 (+ 1161us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:23:19.560331 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:23:19.561607 (+ 1276us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:23:19.561610 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:23:19.562374 (+ 764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:23:19.562375 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:23:19.562465 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:23:19.562478 (+ 13us) client_negotiation.cc:769] Sending connection context
0809 16:23:19.562547 (+ 69us) client_negotiation.cc:240] Negotiation successful
0809 16:23:19.562602 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":362,"mutex_wait_us":75,"thread_start_us":69,"threads_started":1}
I20260809 16:23:19.562711 13933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:23:19.558395 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48312 (local address 127.21.51.131:40517)
0809 16:23:19.558917 (+ 522us) server_negotiation.cc:206] Beginning negotiation
0809 16:23:19.558922 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 16:23:19.558933 (+ 11us) server_negotiation.cc:407] Connection header received
0809 16:23:19.558975 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:23:19.558977 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:23:19.558994 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:23:19.559244 (+ 250us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:23:19.559328 (+ 84us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:23:19.560219 (+ 891us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:23:19.561705 (+ 1486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:23:19.562278 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:23:19.562373 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:23:19.562434 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0809 16:23:19.562571 (+ 137us) server_negotiation.cc:299] Negotiation successful
0809 16:23:19.562610 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":486,"thread_start_us":131,"threads_started":1}
I20260809 16:23:19.562928 13928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:23:19.558451 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.129:37335 (local address 127.0.0.1:59982)
0809 16:23:19.558784 (+ 333us) negotiation.cc:107] Waiting for socket to connect
0809 16:23:19.558792 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0809 16:23:19.558856 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:23:19.559460 (+ 604us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:23:19.559461 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:23:19.559463 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:23:19.559507 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:23:19.559508 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:23:19.560722 (+ 1214us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:23:19.560723 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:23:19.561826 (+ 1103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:23:19.561828 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:23:19.562737 (+ 909us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:23:19.562738 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:23:19.562790 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:23:19.562794 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 16:23:19.562823 (+ 29us) client_negotiation.cc:240] Negotiation successful
0809 16:23:19.562864 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":323,"thread_start_us":57,"threads_started":1}
I20260809 16:23:19.563040 13934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:23:19.558586 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48268 (local address 127.21.51.130:43191)
0809 16:23:19.558990 (+ 404us) server_negotiation.cc:206] Beginning negotiation
0809 16:23:19.558995 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 16:23:19.559005 (+ 10us) server_negotiation.cc:407] Connection header received
0809 16:23:19.559045 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:23:19.559048 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:23:19.559063 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:23:19.559142 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:23:19.559275 (+ 133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:23:19.560404 (+ 1129us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:23:19.561692 (+ 1288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:23:19.562387 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:23:19.562463 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:23:19.562516 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0809 16:23:19.562901 (+ 385us) server_negotiation.cc:299] Negotiation successful
0809 16:23:19.562944 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":370,"thread_start_us":90,"threads_started":1}
I20260809 16:23:19.562827 13930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:23:19.558238 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.130:43191 (local address 127.0.0.1:48268)
0809 16:23:19.558559 (+ 321us) negotiation.cc:107] Waiting for socket to connect
0809 16:23:19.558603 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 16:23:19.558672 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:23:19.559137 (+ 465us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:23:19.559138 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:23:19.559140 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:23:19.559189 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:23:19.559191 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:23:19.560518 (+ 1327us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:23:19.560519 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:23:19.561586 (+ 1067us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:23:19.561589 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:23:19.562554 (+ 965us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:23:19.562555 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:23:19.562619 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:23:19.562624 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 16:23:19.562673 (+ 49us) client_negotiation.cc:240] Negotiation successful
0809 16:23:19.562746 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":292,"thread_start_us":67,"threads_started":1}
I20260809 16:23:19.563436 13935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:23:19.558591 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59982 (local address 127.21.51.129:37335)
0809 16:23:19.559284 (+ 693us) server_negotiation.cc:206] Beginning negotiation
0809 16:23:19.559288 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 16:23:19.559297 (+ 9us) server_negotiation.cc:407] Connection header received
0809 16:23:19.559335 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:23:19.559338 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:23:19.559353 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:23:19.559441 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:23:19.559576 (+ 135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:23:19.560585 (+ 1009us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:23:19.561918 (+ 1333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:23:19.562610 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:23:19.563207 (+ 597us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:23:19.563278 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0809 16:23:19.563303 (+ 25us) server_negotiation.cc:299] Negotiation successful
0809 16:23:19.563347 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":652,"thread_start_us":102,"threads_started":1}
I20260809 16:23:19.563630 13807 tablet_service.cc:1467] Processing CreateTablet for tablet c25966b13f9549ac9ab0b056154c208e (DEFAULT_TABLE table=test-table [id=e312ae0516734d479b150998b4391a01]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:23:19.564026 13807 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c25966b13f9549ac9ab0b056154c208e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:23:19.564090 13400 tablet_service.cc:1467] Processing CreateTablet for tablet c25966b13f9549ac9ab0b056154c208e (DEFAULT_TABLE table=test-table [id=e312ae0516734d479b150998b4391a01]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:23:19.564407 13400 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c25966b13f9549ac9ab0b056154c208e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:23:19.565264 13592 tablet_service.cc:1467] Processing CreateTablet for tablet c25966b13f9549ac9ab0b056154c208e (DEFAULT_TABLE table=test-table [id=e312ae0516734d479b150998b4391a01]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:23:19.565585 13592 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c25966b13f9549ac9ab0b056154c208e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:23:19.567507 13936 tablet_bootstrap.cc:492] T c25966b13f9549ac9ab0b056154c208e P 6aa3f90a2d2042e08594f1dd05d7fe30: Bootstrap starting.
I20260809 16:23:19.567785 13937 tablet_bootstrap.cc:492] T c25966b13f9549ac9ab0b056154c208e P 7d67bdf64e544c339ca52e32fcd4a5a7: Bootstrap starting.
I20260809 16:23:19.568383 13937 tablet_bootstrap.cc:654] T c25966b13f9549ac9ab0b056154c208e P 7d67bdf64e544c339ca52e32fcd4a5a7: Neither blocks nor log segments found. Creating new log.
I20260809 16:23:19.568382 13936 tablet_bootstrap.cc:654] T c25966b13f9549ac9ab0b056154c208e P 6aa3f90a2d2042e08594f1dd05d7fe30: Neither blocks nor log segments found. Creating new log.
I20260809 16:23:19.568439 13939 tablet_bootstrap.cc:492] T c25966b13f9549ac9ab0b056154c208e P 108d298be817495d987fd45a969d96ba: Bootstrap starting.
I20260809 16:23:19.569022 13936 log.cc:824] T c25966b13f9549ac9ab0b056154c208e P 6aa3f90a2d2042e08594f1dd05d7fe30: Log is configured to *not* fsync() on all Append() calls
I20260809 16:23:19.569059 13937 log.cc:824] T c25966b13f9549ac9ab0b056154c208e P 7d67bdf64e544c339ca52e32fcd4a5a7: Log is configured to *not* fsync() on all Append() calls
I20260809 16:23:19.569229 13939 tablet_bootstrap.cc:654] T c25966b13f9549ac9ab0b056154c208e P 108d298be817495d987fd45a969d96ba: Neither blocks nor log segments found. Creating new log.
I20260809 16:23:19.569797 13936 tablet_bootstrap.cc:492] T c25966b13f9549ac9ab0b056154c208e P 6aa3f90a2d2042e08594f1dd05d7fe30: No bootstrap required, opened a new log
I20260809 16:23:19.569815 13939 log.cc:824] T c25966b13f9549ac9ab0b056154c208e P 108d298be817495d987fd45a969d96ba: Log is configured to *not* fsync() on all Append() calls
I20260809 16:23:19.569854 13937 tablet_bootstrap.cc:492] T c25966b13f9549ac9ab0b056154c208e P 7d67bdf64e544c339ca52e32fcd4a5a7: No bootstrap required, opened a new log
I20260809 16:23:19.569931 13937 ts_tablet_manager.cc:1397] T c25966b13f9549ac9ab0b056154c208e P 7d67bdf64e544c339ca52e32fcd4a5a7: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260809 16:23:19.569936 13936 ts_tablet_manager.cc:1397] T c25966b13f9549ac9ab0b056154c208e P 6aa3f90a2d2042e08594f1dd05d7fe30: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 16:23:19.570639 13939 tablet_bootstrap.cc:492] T c25966b13f9549ac9ab0b056154c208e P 108d298be817495d987fd45a969d96ba: No bootstrap required, opened a new log
I20260809 16:23:19.570710 13939 ts_tablet_manager.cc:1397] T c25966b13f9549ac9ab0b056154c208e P 108d298be817495d987fd45a969d96ba: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 16:23:19.571467 13936 raft_consensus.cc:348] T c25966b13f9549ac9ab0b056154c208e P 6aa3f90a2d2042e08594f1dd05d7fe30 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6aa3f90a2d2042e08594f1dd05d7fe30" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 40517 } } peers { permanent_uuid: "108d298be817495d987fd45a969d96ba" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 43191 } } peers { permanent_uuid: "7d67bdf64e544c339ca52e32fcd4a5a7" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 37335 } }
I20260809 16:23:19.571635 13936 raft_consensus.cc:374] T c25966b13f9549ac9ab0b056154c208e P 6aa3f90a2d2042e08594f1dd05d7fe30 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:23:19.571681 13936 raft_consensus.cc:729] T c25966b13f9549ac9ab0b056154c208e P 6aa3f90a2d2042e08594f1dd05d7fe30 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6aa3f90a2d2042e08594f1dd05d7fe30, State: Initialized, Role: FOLLOWER
I20260809 16:23:19.571794 13936 consensus_queue.cc:260] T c25966b13f9549ac9ab0b056154c208e P 6aa3f90a2d2042e08594f1dd05d7fe30 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6aa3f90a2d2042e08594f1dd05d7fe30" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 40517 } } peers { permanent_uuid: "108d298be817495d987fd45a969d96ba" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 43191 } } peers { permanent_uuid: "7d67bdf64e544c339ca52e32fcd4a5a7" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 37335 } }
I20260809 16:23:19.571854 13937 raft_consensus.cc:348] T c25966b13f9549ac9ab0b056154c208e P 7d67bdf64e544c339ca52e32fcd4a5a7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6aa3f90a2d2042e08594f1dd05d7fe30" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 40517 } } peers { permanent_uuid: "108d298be817495d987fd45a969d96ba" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 43191 } } peers { permanent_uuid: "7d67bdf64e544c339ca52e32fcd4a5a7" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 37335 } }
I20260809 16:23:19.572026 13937 raft_consensus.cc:374] T c25966b13f9549ac9ab0b056154c208e P 7d67bdf64e544c339ca52e32fcd4a5a7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:23:19.572064 13937 raft_consensus.cc:729] T c25966b13f9549ac9ab0b056154c208e P 7d67bdf64e544c339ca52e32fcd4a5a7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7d67bdf64e544c339ca52e32fcd4a5a7, State: Initialized, Role: FOLLOWER
I20260809 16:23:19.572098 13936 ts_tablet_manager.cc:1428] T c25966b13f9549ac9ab0b056154c208e P 6aa3f90a2d2042e08594f1dd05d7fe30: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260809 16:23:19.572173 13937 consensus_queue.cc:260] T c25966b13f9549ac9ab0b056154c208e P 7d67bdf64e544c339ca52e32fcd4a5a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6aa3f90a2d2042e08594f1dd05d7fe30" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 40517 } } peers { permanent_uuid: "108d298be817495d987fd45a969d96ba" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 43191 } } peers { permanent_uuid: "7d67bdf64e544c339ca52e32fcd4a5a7" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 37335 } }
I20260809 16:23:19.572432 13937 ts_tablet_manager.cc:1428] T c25966b13f9549ac9ab0b056154c208e P 7d67bdf64e544c339ca52e32fcd4a5a7: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260809 16:23:19.572510 13500 heartbeater.cc:503] Master 127.21.51.190:44381 was elected leader, sending a full tablet report...
I20260809 16:23:19.572569 13910 heartbeater.cc:503] Master 127.21.51.190:44381 was elected leader, sending a full tablet report...
I20260809 16:23:19.572821 13939 raft_consensus.cc:348] T c25966b13f9549ac9ab0b056154c208e P 108d298be817495d987fd45a969d96ba [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6aa3f90a2d2042e08594f1dd05d7fe30" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 40517 } } peers { permanent_uuid: "108d298be817495d987fd45a969d96ba" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 43191 } } peers { permanent_uuid: "7d67bdf64e544c339ca52e32fcd4a5a7" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 37335 } }
I20260809 16:23:19.572986 13939 raft_consensus.cc:374] T c25966b13f9549ac9ab0b056154c208e P 108d298be817495d987fd45a969d96ba [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:23:19.573011 13939 raft_consensus.cc:729] T c25966b13f9549ac9ab0b056154c208e P 108d298be817495d987fd45a969d96ba [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 108d298be817495d987fd45a969d96ba, State: Initialized, Role: FOLLOWER
I20260809 16:23:19.573143 13939 consensus_queue.cc:260] T c25966b13f9549ac9ab0b056154c208e P 108d298be817495d987fd45a969d96ba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6aa3f90a2d2042e08594f1dd05d7fe30" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 40517 } } peers { permanent_uuid: "108d298be817495d987fd45a969d96ba" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 43191 } } peers { permanent_uuid: "7d67bdf64e544c339ca52e32fcd4a5a7" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 37335 } }
I20260809 16:23:19.573426 13939 ts_tablet_manager.cc:1428] T c25966b13f9549ac9ab0b056154c208e P 108d298be817495d987fd45a969d96ba: Time spent starting tablet: real 0.003s user 0.001s sys 0.002s
I20260809 16:23:19.573590 13701 heartbeater.cc:503] Master 127.21.51.190:44381 was elected leader, sending a full tablet report...
I20260809 16:23:19.589404 13947 raft_consensus.cc:482] T c25966b13f9549ac9ab0b056154c208e P 108d298be817495d987fd45a969d96ba [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 16:23:19.589478 13947 raft_consensus.cc:504] T c25966b13f9549ac9ab0b056154c208e P 108d298be817495d987fd45a969d96ba [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6aa3f90a2d2042e08594f1dd05d7fe30" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 40517 } } peers { permanent_uuid: "108d298be817495d987fd45a969d96ba" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 43191 } } peers { permanent_uuid: "7d67bdf64e544c339ca52e32fcd4a5a7" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 37335 } }
I20260809 16:23:19.589749 13947 leader_election.cc:288] T c25966b13f9549ac9ab0b056154c208e P 108d298be817495d987fd45a969d96ba [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6aa3f90a2d2042e08594f1dd05d7fe30 (127.21.51.131:40517), 7d67bdf64e544c339ca52e32fcd4a5a7 (127.21.51.129:37335)
I20260809 16:23:19.592612 13704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:23:19.589939 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.131:40517 (local address 127.21.51.130:48703)
0809 16:23:19.590021 (+ 82us) negotiation.cc:107] Waiting for socket to connect
0809 16:23:19.590050 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0809 16:23:19.590101 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:23:19.590247 (+ 146us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:23:19.590248 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:23:19.590249 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:23:19.590345 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:23:19.590347 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:23:19.591019 (+ 672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:23:19.591020 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:23:19.591780 (+ 760us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:23:19.591782 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:23:19.592377 (+ 595us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:23:19.592378 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:23:19.592461 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:23:19.592466 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 16:23:19.592520 (+ 54us) client_negotiation.cc:240] Negotiation successful
0809 16:23:19.592563 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":56}
I20260809 16:23:19.592655 13933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:23:19.590013 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.51.130:48703 (local address 127.21.51.131:40517)
0809 16:23:19.590071 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0809 16:23:19.590073 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:23:19.590085 (+ 12us) server_negotiation.cc:407] Connection header received
0809 16:23:19.590163 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:23:19.590163 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:23:19.590174 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:23:19.590224 (+ 50us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:23:19.590415 (+ 191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:23:19.590940 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:23:19.591891 (+ 951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:23:19.592295 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:23:19.592351 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:23:19.592358 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0809 16:23:19.592537 (+ 179us) server_negotiation.cc:299] Negotiation successful
0809 16:23:19.592575 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260809 16:23:19.592844 13935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:23:19.590150 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.51.130:57673 (local address 127.21.51.129:37335)
0809 16:23:19.590201 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0809 16:23:19.590203 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:23:19.590303 (+ 100us) server_negotiation.cc:407] Connection header received
0809 16:23:19.590331 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:23:19.590332 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:23:19.590342 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:23:19.590402 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:23:19.590557 (+ 155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:23:19.591059 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:23:19.592001 (+ 942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:23:19.592412 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:23:19.592486 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:23:19.592495 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0809 16:23:19.592701 (+ 206us) server_negotiation.cc:299] Negotiation successful
0809 16:23:19.592749 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260809 16:23:19.593006 13836 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c25966b13f9549ac9ab0b056154c208e" candidate_uuid: "108d298be817495d987fd45a969d96ba" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6aa3f90a2d2042e08594f1dd05d7fe30" is_pre_election: true
I20260809 16:23:19.593120 13948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:23:19.590079 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.129:37335 (local address 127.21.51.130:57673)
0809 16:23:19.590217 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0809 16:23:19.590226 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 16:23:19.590276 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:23:19.590425 (+ 149us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:23:19.590426 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:23:19.590428 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:23:19.590491 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:23:19.590492 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:23:19.591141 (+ 649us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:23:19.591141 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:23:19.591915 (+ 774us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:23:19.591917 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:23:19.592520 (+ 603us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:23:19.592521 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:23:19.592593 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:23:19.592598 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 16:23:19.592656 (+ 58us) client_negotiation.cc:240] Negotiation successful
0809 16:23:19.593036 (+ 380us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":90,"threads_started":1}
I20260809 16:23:19.593119 13836 raft_consensus.cc:2454] T c25966b13f9549ac9ab0b056154c208e P 6aa3f90a2d2042e08594f1dd05d7fe30 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 108d298be817495d987fd45a969d96ba in term 0.
I20260809 16:23:19.593330 13538 leader_election.cc:302] T c25966b13f9549ac9ab0b056154c208e P 108d298be817495d987fd45a969d96ba [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 108d298be817495d987fd45a969d96ba, 6aa3f90a2d2042e08594f1dd05d7fe30; no voters:
I20260809 16:23:19.593395 13427 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c25966b13f9549ac9ab0b056154c208e" candidate_uuid: "108d298be817495d987fd45a969d96ba" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7d67bdf64e544c339ca52e32fcd4a5a7" is_pre_election: true
I20260809 16:23:19.593504 13947 raft_consensus.cc:2781] T c25966b13f9549ac9ab0b056154c208e P 108d298be817495d987fd45a969d96ba [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 16:23:19.593530 13427 raft_consensus.cc:2454] T c25966b13f9549ac9ab0b056154c208e P 7d67bdf64e544c339ca52e32fcd4a5a7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 108d298be817495d987fd45a969d96ba in term 0.
I20260809 16:23:19.593572 13947 raft_consensus.cc:482] T c25966b13f9549ac9ab0b056154c208e P 108d298be817495d987fd45a969d96ba [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 16:23:19.593596 13947 raft_consensus.cc:3037] T c25966b13f9549ac9ab0b056154c208e P 108d298be817495d987fd45a969d96ba [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:23:19.594278 13947 raft_consensus.cc:504] T c25966b13f9549ac9ab0b056154c208e P 108d298be817495d987fd45a969d96ba [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6aa3f90a2d2042e08594f1dd05d7fe30" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 40517 } } peers { permanent_uuid: "108d298be817495d987fd45a969d96ba" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 43191 } } peers { permanent_uuid: "7d67bdf64e544c339ca52e32fcd4a5a7" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 37335 } }
I20260809 16:23:19.594408 13947 leader_election.cc:288] T c25966b13f9549ac9ab0b056154c208e P 108d298be817495d987fd45a969d96ba [CANDIDATE]: Term 1 election: Requested vote from peers 6aa3f90a2d2042e08594f1dd05d7fe30 (127.21.51.131:40517), 7d67bdf64e544c339ca52e32fcd4a5a7 (127.21.51.129:37335)
I20260809 16:23:19.594535 13836 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c25966b13f9549ac9ab0b056154c208e" candidate_uuid: "108d298be817495d987fd45a969d96ba" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6aa3f90a2d2042e08594f1dd05d7fe30"
I20260809 16:23:19.594635 13836 raft_consensus.cc:3037] T c25966b13f9549ac9ab0b056154c208e P 6aa3f90a2d2042e08594f1dd05d7fe30 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:23:19.594614 13427 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c25966b13f9549ac9ab0b056154c208e" candidate_uuid: "108d298be817495d987fd45a969d96ba" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7d67bdf64e544c339ca52e32fcd4a5a7"
I20260809 16:23:19.594707 13427 raft_consensus.cc:3037] T c25966b13f9549ac9ab0b056154c208e P 7d67bdf64e544c339ca52e32fcd4a5a7 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:23:19.595319 13836 raft_consensus.cc:2454] T c25966b13f9549ac9ab0b056154c208e P 6aa3f90a2d2042e08594f1dd05d7fe30 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 108d298be817495d987fd45a969d96ba in term 1.
I20260809 16:23:19.595396 13427 raft_consensus.cc:2454] T c25966b13f9549ac9ab0b056154c208e P 7d67bdf64e544c339ca52e32fcd4a5a7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 108d298be817495d987fd45a969d96ba in term 1.
I20260809 16:23:19.595516 13538 leader_election.cc:302] T c25966b13f9549ac9ab0b056154c208e P 108d298be817495d987fd45a969d96ba [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 108d298be817495d987fd45a969d96ba, 6aa3f90a2d2042e08594f1dd05d7fe30; no voters:
I20260809 16:23:19.595655 13947 raft_consensus.cc:2781] T c25966b13f9549ac9ab0b056154c208e P 108d298be817495d987fd45a969d96ba [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:23:19.595822 13947 raft_consensus.cc:686] T c25966b13f9549ac9ab0b056154c208e P 108d298be817495d987fd45a969d96ba [term 1 LEADER]: Becoming Leader. State: Replica: 108d298be817495d987fd45a969d96ba, State: Running, Role: LEADER
I20260809 16:23:19.595914 13947 consensus_queue.cc:237] T c25966b13f9549ac9ab0b056154c208e P 108d298be817495d987fd45a969d96ba [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6aa3f90a2d2042e08594f1dd05d7fe30" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 40517 } } peers { permanent_uuid: "108d298be817495d987fd45a969d96ba" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 43191 } } peers { permanent_uuid: "7d67bdf64e544c339ca52e32fcd4a5a7" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 37335 } }
I20260809 16:23:19.596743 13217 catalog_manager.cc:5495] T c25966b13f9549ac9ab0b056154c208e P 108d298be817495d987fd45a969d96ba reported cstate change: term changed from 0 to 1, leader changed from <none> to 108d298be817495d987fd45a969d96ba (127.21.51.130). New cstate: current_term: 1 leader_uuid: "108d298be817495d987fd45a969d96ba" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6aa3f90a2d2042e08594f1dd05d7fe30" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 40517 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "108d298be817495d987fd45a969d96ba" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 43191 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7d67bdf64e544c339ca52e32fcd4a5a7" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 37335 } health_report { overall_health: UNKNOWN } } }
I20260809 16:23:19.610105 13934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:23:19.607626 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48280 (local address 127.21.51.130:43191)
0809 16:23:19.607684 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0809 16:23:19.607686 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:23:19.607711 (+ 25us) server_negotiation.cc:407] Connection header received
0809 16:23:19.607739 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:23:19.607740 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:23:19.607760 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:23:19.607814 (+ 54us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 16:23:19.607943 (+ 129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:23:19.608397 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:23:19.609290 (+ 893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:23:19.609471 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:23:19.609527 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:23:19.609726 (+ 199us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 16:23:19.609850 (+ 124us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 16:23:19.609916 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0809 16:23:19.610007 (+ 91us) server_negotiation.cc:299] Negotiation successful
0809 16:23:19.610034 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260809 16:23:19.611959 13427 raft_consensus.cc:1261] T c25966b13f9549ac9ab0b056154c208e P 7d67bdf64e544c339ca52e32fcd4a5a7 [term 1 FOLLOWER]: Refusing update from remote peer 108d298be817495d987fd45a969d96ba: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 16:23:19.611987 13836 raft_consensus.cc:1261] T c25966b13f9549ac9ab0b056154c208e P 6aa3f90a2d2042e08594f1dd05d7fe30 [term 1 FOLLOWER]: Refusing update from remote peer 108d298be817495d987fd45a969d96ba: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 16:23:19.612377 13947 consensus_queue.cc:1035] T c25966b13f9549ac9ab0b056154c208e P 108d298be817495d987fd45a969d96ba [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d67bdf64e544c339ca52e32fcd4a5a7" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 37335 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 16:23:19.612541 13949 consensus_queue.cc:1035] T c25966b13f9549ac9ab0b056154c208e P 108d298be817495d987fd45a969d96ba [LEADER]: Connected to new peer: Peer: permanent_uuid: "6aa3f90a2d2042e08594f1dd05d7fe30" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 40517 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 16:23:19.614872 13955 mvcc.cc:205] Tried to move back new op lower bound from 7316654488007573504 to 7316654487945576448. Current Snapshot: MvccSnapshot[applied={T|T < 7316654488007573504}]
I20260809 16:23:19.615492 13960 mvcc.cc:205] Tried to move back new op lower bound from 7316654488007573504 to 7316654487945576448. Current Snapshot: MvccSnapshot[applied={T|T < 7316654488007573504}]
I20260809 16:23:19.615502 13958 mvcc.cc:205] Tried to move back new op lower bound from 7316654488007573504 to 7316654487945576448. Current Snapshot: MvccSnapshot[applied={T|T < 7316654488007573504}]
I20260809 16:23:19.617411 13217 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41496:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 16:23:19.617509 13217 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41496:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 16:23:19.618377 13217 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8c44acc0f3e8446693dc64d77163f6b4: Sending DeleteTablet for 3 replicas of tablet c25966b13f9549ac9ab0b056154c208e
I20260809 16:23:19.618711 21710 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrNqUCn/build/release/bin/kudu with pid 13316
I20260809 16:23:19.618706 13592 tablet_service.cc:1514] Processing DeleteTablet for tablet c25966b13f9549ac9ab0b056154c208e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:23:19 UTC) from {username='kudu', principal='kudu/127.21.51.190@KRBTEST.COM'} at 127.0.0.1:48268
I20260809 16:23:19.618808 13807 tablet_service.cc:1514] Processing DeleteTablet for tablet c25966b13f9549ac9ab0b056154c208e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:23:19 UTC) from {username='kudu', principal='kudu/127.21.51.190@KRBTEST.COM'} at 127.0.0.1:48312
I20260809 16:23:19.618837 13400 tablet_service.cc:1514] Processing DeleteTablet for tablet c25966b13f9549ac9ab0b056154c208e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:23:19 UTC) from {username='kudu', principal='kudu/127.21.51.190@KRBTEST.COM'} at 127.0.0.1:59982
I20260809 16:23:19.625123 13975 tablet_replica.cc:330] T c25966b13f9549ac9ab0b056154c208e P 108d298be817495d987fd45a969d96ba: stopping tablet replica
I20260809 16:23:19.625272 13975 raft_consensus.cc:2229] T c25966b13f9549ac9ab0b056154c208e P 108d298be817495d987fd45a969d96ba [term 1 LEADER]: Raft consensus shutting down.
I20260809 16:23:19.625448 13975 raft_consensus.cc:2258] T c25966b13f9549ac9ab0b056154c208e P 108d298be817495d987fd45a969d96ba [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 16:23:19.625993 13975 ts_tablet_manager.cc:1905] T c25966b13f9549ac9ab0b056154c208e P 108d298be817495d987fd45a969d96ba: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 16:23:19.627838 13975 ts_tablet_manager.cc:1918] T c25966b13f9549ac9ab0b056154c208e P 108d298be817495d987fd45a969d96ba: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 16:23:19.627954 13975 log.cc:1196] T c25966b13f9549ac9ab0b056154c208e P 108d298be817495d987fd45a969d96ba: Deleting WAL directory at /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292417722232-21710-0/minicluster-data/ts-1/wal/wals/c25966b13f9549ac9ab0b056154c208e
I20260809 16:23:19.628276 13975 ts_tablet_manager.cc:1939] T c25966b13f9549ac9ab0b056154c208e P 108d298be817495d987fd45a969d96ba: Deleting consensus metadata
I20260809 16:23:19.628786 13197 catalog_manager.cc:4841] TS 108d298be817495d987fd45a969d96ba (127.21.51.130:43191): tablet c25966b13f9549ac9ab0b056154c208e (table test-table [id=e312ae0516734d479b150998b4391a01]) successfully deleted
I20260809 16:23:19.628911 13976 tablet_replica.cc:330] T c25966b13f9549ac9ab0b056154c208e P 6aa3f90a2d2042e08594f1dd05d7fe30: stopping tablet replica
I20260809 16:23:19.629043 13976 raft_consensus.cc:2229] T c25966b13f9549ac9ab0b056154c208e P 6aa3f90a2d2042e08594f1dd05d7fe30 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 16:23:19.629163 13976 raft_consensus.cc:2258] T c25966b13f9549ac9ab0b056154c208e P 6aa3f90a2d2042e08594f1dd05d7fe30 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 16:23:19.629626 13976 ts_tablet_manager.cc:1905] T c25966b13f9549ac9ab0b056154c208e P 6aa3f90a2d2042e08594f1dd05d7fe30: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260809 16:23:19.631373 13199 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.21.51.129:37335 (error 108)
I20260809 16:23:19.631524 13976 ts_tablet_manager.cc:1918] T c25966b13f9549ac9ab0b056154c208e P 6aa3f90a2d2042e08594f1dd05d7fe30: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 16:23:19.631603 13976 log.cc:1196] T c25966b13f9549ac9ab0b056154c208e P 6aa3f90a2d2042e08594f1dd05d7fe30: Deleting WAL directory at /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292417722232-21710-0/minicluster-data/ts-2/wal/wals/c25966b13f9549ac9ab0b056154c208e
I20260809 16:23:19.631884 13976 ts_tablet_manager.cc:1939] T c25966b13f9549ac9ab0b056154c208e P 6aa3f90a2d2042e08594f1dd05d7fe30: Deleting consensus metadata
I20260809 16:23:19.632017 21710 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrNqUCn/build/release/bin/kudu with pid 13510
I20260809 16:23:19.641927 13199 catalog_manager.cc:4841] TS 6aa3f90a2d2042e08594f1dd05d7fe30 (127.21.51.131:40517): tablet c25966b13f9549ac9ab0b056154c208e (table test-table [id=e312ae0516734d479b150998b4391a01]) successfully deleted
I20260809 16:23:19.645103 21710 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrNqUCn/build/release/bin/kudu with pid 13712
I20260809 16:23:19.647562 13928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:23:19.645282 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.129:37335 (local address 127.0.0.1:59992)
0809 16:23:19.647463 (+ 2181us) negotiation.cc:107] Waiting for socket to connect
0809 16:23:19.647514 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.51.129:37335: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2149}
W20260809 16:23:19.648522 13199 catalog_manager.cc:4584] TS 7d67bdf64e544c339ca52e32fcd4a5a7 (127.21.51.129:37335): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet c25966b13f9549ac9ab0b056154c208e: Network error: Client connection negotiation failed: client connection to 127.21.51.129:37335: connect: Connection refused (error 111)
I20260809 16:23:19.656261 21710 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrNqUCn/build/release/bin/kudu with pid 13174
I20260809 16:23:39.665869 21710 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrNqUCn/build/release/bin/kudu
/tmp/dist-test-taskrNqUCn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292417722232-21710-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292417722232-21710-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292417722232-21710-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292417722232-21710-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.51.190:44381
--webserver_interface=127.21.51.190
--webserver_port=34771
--keytab_file=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.190.keytab
--principal=kudu/127.21.51.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.51.148:34103
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.51.190:44381
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292417722232-21710-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292417722232-21710-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292417722232-21710-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:23:39.749547 13978 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:23:39.749752 13978 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:23:39.749775 13978 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:23:39.750918 13978 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260809 16:23:39.750959 13978 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260809 16:23:39.750973 13978 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260809 16:23:39.750985 13978 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 16:23:39.751037 13978 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:23:39.751055 13978 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:23:39.751076 13978 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 16:23:39.751099 13978 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 16:23:39.752584 13978 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.51.148:34103
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292417722232-21710-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292417722232-21710-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.21.51.190:44381
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.190.keytab
--principal=kudu/127.21.51.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.51.190:44381
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292417722232-21710-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.51.190
--webserver_port=34771
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13978
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292417722232-21710-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:23:39.752862 13978 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:23:39.753105 13978 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:23:39.762869 13990 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:23:39.762835 13993 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:23:39.762881 13989 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:23:39.762948 13978 server_base.cc:1034] running on GCE node
I20260809 16:23:39.763415 13978 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:23:39.763760 13978 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:23:39.764925 13978 hybrid_clock.cc:648] HybridClock initialized: now 1786292619764905 us; error 39 us; skew 500 ppm
Aug 09 16:23:39 dist-test-slave-s11t krb5kdc[13156](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292619, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.51.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:23:39.770920 13978 init.cc:377] Logged in from keytab as kudu/127.21.51.190@KRBTEST.COM (short username kudu)
I20260809 16:23:39.771464 13978 webserver.cc:460] Webserver started at http://127.21.51.190:34771/ using document root <none> and password file <none>
I20260809 16:23:39.771716 13978 fs_manager.cc:359] Metadata directory not provided
I20260809 16:23:39.771811 13978 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:23:39.773319 13978 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20260809 16:23:39.774084 14006 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:23:39.774299 13978 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 16:23:39.774416 13978 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292417722232-21710-0/minicluster-data/master-0/data,/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292417722232-21710-0/minicluster-data/master-0/wal
uuid: "8c44acc0f3e8446693dc64d77163f6b4"
format_stamp: "Formatted at 2026-08-09 16:23:18 on dist-test-slave-s11t"
server_key: "feee4944f16d07b04e52e984838a350c"
server_key_iv: "b5fdc51511f5019e4a395d2478e913f1"
server_key_version: "encryptionkey@0"
I20260809 16:23:39.774837 13978 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292417722232-21710-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292417722232-21710-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292417722232-21710-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:23:39.791944 13978 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:23:39.794306 13978 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:23:39.794484 13978 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:23:39.798959 13978 rpc_server.cc:307] RPC server started. Bound to: 127.21.51.190:44381
I20260809 16:23:39.799021 14081 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.51.190:44381 every 8 connection(s)
I20260809 16:23:39.799422 13978 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292417722232-21710-0/minicluster-data/master-0/data/info.pb
I20260809 16:23:39.800998 14082 sys_catalog.cc:263] Verifying existing consensus state
I20260809 16:23:39.801671 14082 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8c44acc0f3e8446693dc64d77163f6b4: Bootstrap starting.
I20260809 16:23:39.801882 21710 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrNqUCn/build/release/bin/kudu as pid 13978
I20260809 16:23:39.802168 21710 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrNqUCn/build/release/bin/kudu
/tmp/dist-test-taskrNqUCn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292417722232-21710-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292417722232-21710-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292417722232-21710-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292417722232-21710-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.51.129:37335
--local_ip_for_outbound_sockets=127.21.51.129
--tserver_master_addrs=127.21.51.190:44381
--webserver_port=40013
--webserver_interface=127.21.51.129
--keytab_file=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.129.keytab
--principal=kudu/127.21.51.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.51.148:34103
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292417722232-21710-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292417722232-21710-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292417722232-21710-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 16:23:39.810461 14082 log.cc:824] T 00000000000000000000000000000000 P 8c44acc0f3e8446693dc64d77163f6b4: Log is configured to *not* fsync() on all Append() calls
I20260809 16:23:39.812398 14082 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8c44acc0f3e8446693dc64d77163f6b4: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 16:23:39.812796 14082 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8c44acc0f3e8446693dc64d77163f6b4: Bootstrap complete.
I20260809 16:23:39.815532 14082 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8c44acc0f3e8446693dc64d77163f6b4 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c44acc0f3e8446693dc64d77163f6b4" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 44381 } }
I20260809 16:23:39.815762 14082 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8c44acc0f3e8446693dc64d77163f6b4 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c44acc0f3e8446693dc64d77163f6b4, State: Initialized, Role: FOLLOWER
I20260809 16:23:39.815933 14082 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8c44acc0f3e8446693dc64d77163f6b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c44acc0f3e8446693dc64d77163f6b4" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 44381 } }
I20260809 16:23:39.815994 14082 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8c44acc0f3e8446693dc64d77163f6b4 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 16:23:39.816061 14082 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8c44acc0f3e8446693dc64d77163f6b4 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 16:23:39.816133 14082 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8c44acc0f3e8446693dc64d77163f6b4 [term 1 FOLLOWER]: Advancing to term 2
I20260809 16:23:39.817317 14082 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8c44acc0f3e8446693dc64d77163f6b4 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c44acc0f3e8446693dc64d77163f6b4" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 44381 } }
I20260809 16:23:39.817404 14082 leader_election.cc:302] T 00000000000000000000000000000000 P 8c44acc0f3e8446693dc64d77163f6b4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8c44acc0f3e8446693dc64d77163f6b4; no voters:
I20260809 16:23:39.817600 14082 leader_election.cc:288] T 00000000000000000000000000000000 P 8c44acc0f3e8446693dc64d77163f6b4 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260809 16:23:39.817639 14087 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8c44acc0f3e8446693dc64d77163f6b4 [term 2 FOLLOWER]: Leader election won for term 2
I20260809 16:23:39.817827 14087 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8c44acc0f3e8446693dc64d77163f6b4 [term 2 LEADER]: Becoming Leader. State: Replica: 8c44acc0f3e8446693dc64d77163f6b4, State: Running, Role: LEADER
I20260809 16:23:39.817884 14082 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8c44acc0f3e8446693dc64d77163f6b4 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 16:23:39.818548 14087 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8c44acc0f3e8446693dc64d77163f6b4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c44acc0f3e8446693dc64d77163f6b4" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 44381 } }
I20260809 16:23:39.819130 14090 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8c44acc0f3e8446693dc64d77163f6b4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "8c44acc0f3e8446693dc64d77163f6b4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c44acc0f3e8446693dc64d77163f6b4" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 44381 } } }
I20260809 16:23:39.819231 14090 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8c44acc0f3e8446693dc64d77163f6b4 [sys.catalog]: This master's current role is: LEADER
I20260809 16:23:39.819625 14088 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8c44acc0f3e8446693dc64d77163f6b4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8c44acc0f3e8446693dc64d77163f6b4. Latest consensus state: current_term: 2 leader_uuid: "8c44acc0f3e8446693dc64d77163f6b4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c44acc0f3e8446693dc64d77163f6b4" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 44381 } } }
I20260809 16:23:39.819681 14088 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8c44acc0f3e8446693dc64d77163f6b4 [sys.catalog]: This master's current role is: LEADER
I20260809 16:23:39.819749 14110 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 16:23:39.820523 14110 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 16:23:39.820627 14110 catalog_manager.cc:1259] Loaded cluster ID: ae92579412e5493faccfdf6bb3f0cb1f
I20260809 16:23:39.820660 14110 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 16:23:39.821954 14110 catalog_manager.cc:1504] Loading token signing keys...
I20260809 16:23:39.822142 14110 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 8c44acc0f3e8446693dc64d77163f6b4: Loaded TSK: 0
I20260809 16:23:39.832098 14110 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8c44acc0f3e8446693dc64d77163f6b4: Generated new TSK 1
I20260809 16:23:39.832190 14110 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260809 16:23:39.890034 14085 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:23:39.890268 14085 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:23:39.890311 14085 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:23:39.891474 14085 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:23:39.891541 14085 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:23:39.891587 14085 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.51.129
I20260809 16:23:39.893133 14085 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.51.148:34103
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292417722232-21710-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292417722232-21710-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.129.keytab
--principal=kudu/127.21.51.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.51.129:37335
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292417722232-21710-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.51.129
--webserver_port=40013
--tserver_master_addrs=127.21.51.190:44381
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14085
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.51.129
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292417722232-21710-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:23:39.893385 14085 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:23:39.893620 14085 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:23:39.899544 14122 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:23:39.899559 14120 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:23:39.899564 14119 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:23:39.899821 14085 server_base.cc:1034] running on GCE node
I20260809 16:23:39.900036 14085 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:23:39.900290 14085 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:23:39.901439 14085 hybrid_clock.cc:648] HybridClock initialized: now 1786292619901416 us; error 39 us; skew 500 ppm
Aug 09 16:23:39 dist-test-slave-s11t krb5kdc[13156](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292619, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.51.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:23:39.903966 14085 init.cc:377] Logged in from keytab as kudu/127.21.51.129@KRBTEST.COM (short username kudu)
I20260809 16:23:39.904438 14085 webserver.cc:460] Webserver started at http://127.21.51.129:40013/ using document root <none> and password file <none>
I20260809 16:23:39.904723 14085 fs_manager.cc:359] Metadata directory not provided
I20260809 16:23:39.904824 14085 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:23:39.906169 14085 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:23:39.906903 14130 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:23:39.907117 14085 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:23:39.907215 14085 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292417722232-21710-0/minicluster-data/ts-0/data,/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292417722232-21710-0/minicluster-data/ts-0/wal
uuid: "7d67bdf64e544c339ca52e32fcd4a5a7"
format_stamp: "Formatted at 2026-08-09 16:23:19 on dist-test-slave-s11t"
server_key: "3899e01c0413848a2deae05fbcc2cec1"
server_key_iv: "6f19b55e88ae041ff9a866bc626841a2"
server_key_version: "encryptionkey@0"
I20260809 16:23:39.907564 14085 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292417722232-21710-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292417722232-21710-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292417722232-21710-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:23:39.925482 14085 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:23:39.927610 14085 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:23:39.927772 14085 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:23:39.928009 14085 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:23:39.928457 14140 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260809 16:23:39.930382 14085 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260809 16:23:39.930445 14085 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260809 16:23:39.930488 14085 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260809 16:23:39.931092 14085 ts_tablet_manager.cc:610] Registered 1 tablets
I20260809 16:23:39.931150 14085 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260809 16:23:39.931229 14140 tablet_bootstrap.cc:492] T c25966b13f9549ac9ab0b056154c208e P 7d67bdf64e544c339ca52e32fcd4a5a7: Bootstrap starting.
I20260809 16:23:39.938311 14085 rpc_server.cc:307] RPC server started. Bound to: 127.21.51.129:37335
I20260809 16:23:39.938861 14085 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292417722232-21710-0/minicluster-data/ts-0/data/info.pb
I20260809 16:23:39.942822 14140 log.cc:824] T c25966b13f9549ac9ab0b056154c208e P 7d67bdf64e544c339ca52e32fcd4a5a7: Log is configured to *not* fsync() on all Append() calls
I20260809 16:23:39.946336 14140 tablet_bootstrap.cc:492] T c25966b13f9549ac9ab0b056154c208e P 7d67bdf64e544c339ca52e32fcd4a5a7: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 16:23:39.946707 14302 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.51.129:37335 every 8 connection(s)
I20260809 16:23:39.946833 14140 tablet_bootstrap.cc:492] T c25966b13f9549ac9ab0b056154c208e P 7d67bdf64e544c339ca52e32fcd4a5a7: Bootstrap complete.
I20260809 16:23:39.948082 14140 ts_tablet_manager.cc:1397] T c25966b13f9549ac9ab0b056154c208e P 7d67bdf64e544c339ca52e32fcd4a5a7: Time spent bootstrapping tablet: real 0.017s user 0.011s sys 0.003s
I20260809 16:23:39.948388 21710 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrNqUCn/build/release/bin/kudu as pid 14085
I20260809 16:23:39.948738 21710 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrNqUCn/build/release/bin/kudu
/tmp/dist-test-taskrNqUCn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292417722232-21710-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292417722232-21710-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292417722232-21710-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292417722232-21710-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.51.130:43191
--local_ip_for_outbound_sockets=127.21.51.130
--tserver_master_addrs=127.21.51.190:44381
--webserver_port=36157
--webserver_interface=127.21.51.130
--keytab_file=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.130.keytab
--principal=kudu/127.21.51.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.51.148:34103
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292417722232-21710-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292417722232-21710-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292417722232-21710-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 16:23:39.949502 14140 raft_consensus.cc:348] T c25966b13f9549ac9ab0b056154c208e P 7d67bdf64e544c339ca52e32fcd4a5a7 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6aa3f90a2d2042e08594f1dd05d7fe30" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 40517 } } peers { permanent_uuid: "108d298be817495d987fd45a969d96ba" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 43191 } } peers { permanent_uuid: "7d67bdf64e544c339ca52e32fcd4a5a7" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 37335 } }
I20260809 16:23:39.949752 14140 raft_consensus.cc:729] T c25966b13f9549ac9ab0b056154c208e P 7d67bdf64e544c339ca52e32fcd4a5a7 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7d67bdf64e544c339ca52e32fcd4a5a7, State: Initialized, Role: FOLLOWER
I20260809 16:23:39.949894 14140 consensus_queue.cc:260] T c25966b13f9549ac9ab0b056154c208e P 7d67bdf64e544c339ca52e32fcd4a5a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6aa3f90a2d2042e08594f1dd05d7fe30" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 40517 } } peers { permanent_uuid: "108d298be817495d987fd45a969d96ba" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 43191 } } peers { permanent_uuid: "7d67bdf64e544c339ca52e32fcd4a5a7" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 37335 } }
I20260809 16:23:39.950209 14140 ts_tablet_manager.cc:1428] T c25966b13f9549ac9ab0b056154c208e P 7d67bdf64e544c339ca52e32fcd4a5a7: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
Aug 09 16:23:39 dist-test-slave-s11t krb5kdc[13156](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292619, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.51.129@KRBTEST.COM for kudu/127.21.51.190@KRBTEST.COM
I20260809 16:23:39.960609 14310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:23:39.947568 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.51.129:47501 (local address 127.21.51.190:44381)
0809 16:23:39.947911 (+ 343us) server_negotiation.cc:206] Beginning negotiation
0809 16:23:39.947921 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 16:23:39.947961 (+ 40us) server_negotiation.cc:407] Connection header received
0809 16:23:39.950699 (+ 2738us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:23:39.950707 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:23:39.950765 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:23:39.950908 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:23:39.951808 (+ 900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:23:39.952796 (+ 988us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:23:39.953908 (+ 1112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:23:39.954310 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:23:39.954438 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:23:39.956564 (+ 2126us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:23:39.956568 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:23:39.956568 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:23:39.956578 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:23:39.958763 (+ 2185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:23:39.959268 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:23:39.959271 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:23:39.959271 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:23:39.959355 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:23:39.959664 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:23:39.959665 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:23:39.959666 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:23:39.959886 (+ 220us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:23:39.959985 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0809 16:23:39.960290 (+ 305us) server_negotiation.cc:299] Negotiation successful
0809 16:23:39.960418 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":298,"thread_start_us":85,"threads_started":1}
I20260809 16:23:39.960624 14309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:23:39.947265 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:44381 (local address 127.21.51.129:47501)
0809 16:23:39.947434 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0809 16:23:39.947482 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0809 16:23:39.948172 (+ 690us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:23:39.950983 (+ 2811us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:23:39.950988 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:23:39.951372 (+ 384us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:23:39.951667 (+ 295us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:23:39.951673 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:23:39.952908 (+ 1235us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:23:39.952909 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:23:39.953817 (+ 908us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:23:39.953819 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:23:39.954402 (+ 583us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:23:39.954403 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:23:39.954527 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:23:39.955086 (+ 559us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:23:39.955094 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:23:39.956448 (+ 1354us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:23:39.958871 (+ 2423us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:23:39.958879 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:23:39.958881 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:23:39.959185 (+ 304us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:23:39.959446 (+ 261us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:23:39.959447 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:23:39.959447 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:23:39.959557 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:23:39.959977 (+ 420us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:23:39.959984 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:23:39.960118 (+ 134us) client_negotiation.cc:769] Sending connection context
0809 16:23:39.960221 (+ 103us) client_negotiation.cc:240] Negotiation successful
0809 16:23:39.960397 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":72,"threads_started":1}
I20260809 16:23:39.961668 14304 heartbeater.cc:347] Connected to a master server at 127.21.51.190:44381
I20260809 16:23:39.961787 14304 heartbeater.cc:464] Registering TS with master...
I20260809 16:23:39.961964 14304 heartbeater.cc:511] Master 127.21.51.190:44381 requested a full tablet report, sending...
I20260809 16:23:39.962726 14032 ts_manager.cc:194] Registered new tserver with Master: 7d67bdf64e544c339ca52e32fcd4a5a7 (127.21.51.129:37335)
I20260809 16:23:39.963863 14032 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.51.129@KRBTEST.COM'} at 127.21.51.129:47501
I20260809 16:23:39.964665 14304 heartbeater.cc:503] Master 127.21.51.190:44381 was elected leader, sending a full tablet report...
Aug 09 16:23:39 dist-test-slave-s11t krb5kdc[13156](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292619, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.51.190@KRBTEST.COM for kudu/127.21.51.129@KRBTEST.COM
I20260809 16:23:39.973950 14315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:23:39.963314 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.129:37335 (local address 127.0.0.1:59664)
0809 16:23:39.963817 (+ 503us) negotiation.cc:107] Waiting for socket to connect
0809 16:23:39.963849 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0809 16:23:39.963921 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:23:39.965012 (+ 1091us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:23:39.965017 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:23:39.965372 (+ 355us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:23:39.965423 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:23:39.965426 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:23:39.966309 (+ 883us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:23:39.966310 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:23:39.967069 (+ 759us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:23:39.967072 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:23:39.967810 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:23:39.967811 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:23:39.967942 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:23:39.968497 (+ 555us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:23:39.968502 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:23:39.970343 (+ 1841us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:23:39.972515 (+ 2172us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:23:39.972523 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:23:39.972526 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:23:39.972878 (+ 352us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:23:39.973129 (+ 251us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:23:39.973130 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:23:39.973131 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:23:39.973206 (+ 75us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:23:39.973537 (+ 331us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:23:39.973543 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:23:39.973600 (+ 57us) client_negotiation.cc:769] Sending connection context
0809 16:23:39.973708 (+ 108us) client_negotiation.cc:240] Negotiation successful
0809 16:23:39.973836 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":478,"thread_start_us":115,"threads_started":1}
I20260809 16:23:39.974035 14317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:23:39.963576 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59664 (local address 127.21.51.129:37335)
0809 16:23:39.963751 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0809 16:23:39.963755 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 16:23:39.964021 (+ 266us) server_negotiation.cc:407] Connection header received
0809 16:23:39.964205 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:23:39.964208 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:23:39.964223 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:23:39.964952 (+ 729us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:23:39.965519 (+ 567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:23:39.966219 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:23:39.967175 (+ 956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:23:39.967719 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:23:39.967790 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:23:39.970456 (+ 2666us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:23:39.970465 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:23:39.970465 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:23:39.970473 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:23:39.972409 (+ 1936us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:23:39.972984 (+ 575us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:23:39.972987 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:23:39.972987 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:23:39.973056 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:23:39.973272 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:23:39.973273 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:23:39.973273 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:23:39.973428 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:23:39.973505 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0809 16:23:39.973779 (+ 274us) server_negotiation.cc:299] Negotiation successful
0809 16:23:39.973878 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":150,"thread_start_us":66,"threads_started":1}
I20260809 16:23:39.974402 14204 tablet_service.cc:1514] Processing DeleteTablet for tablet c25966b13f9549ac9ab0b056154c208e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:23:19 UTC) from {username='kudu', principal='kudu/127.21.51.190@KRBTEST.COM'} at 127.0.0.1:59664
I20260809 16:23:39.975090 14318 tablet_replica.cc:330] stopping tablet replica
I20260809 16:23:39.975190 14318 raft_consensus.cc:2229] T c25966b13f9549ac9ab0b056154c208e P 7d67bdf64e544c339ca52e32fcd4a5a7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 16:23:39.975251 14318 raft_consensus.cc:2258] T c25966b13f9549ac9ab0b056154c208e P 7d67bdf64e544c339ca52e32fcd4a5a7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 16:23:39.975636 14318 ts_tablet_manager.cc:1905] T c25966b13f9549ac9ab0b056154c208e P 7d67bdf64e544c339ca52e32fcd4a5a7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 16:23:39.978386 14318 ts_tablet_manager.cc:1918] T c25966b13f9549ac9ab0b056154c208e P 7d67bdf64e544c339ca52e32fcd4a5a7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 16:23:39.978480 14318 log.cc:1196] T c25966b13f9549ac9ab0b056154c208e P 7d67bdf64e544c339ca52e32fcd4a5a7: Deleting WAL directory at /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292417722232-21710-0/minicluster-data/ts-0/wal/wals/c25966b13f9549ac9ab0b056154c208e
I20260809 16:23:39.978758 14318 ts_tablet_manager.cc:1939] T c25966b13f9549ac9ab0b056154c208e P 7d67bdf64e544c339ca52e32fcd4a5a7: Deleting consensus metadata
I20260809 16:23:39.979308 14012 catalog_manager.cc:4841] TS 7d67bdf64e544c339ca52e32fcd4a5a7 (127.21.51.129:37335): tablet c25966b13f9549ac9ab0b056154c208e (table test-table [id=e312ae0516734d479b150998b4391a01]) successfully deleted
W20260809 16:23:40.052045 14314 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:23:40.052289 14314 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:23:40.052328 14314 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:23:40.053493 14314 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:23:40.053571 14314 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:23:40.053617 14314 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.51.130
I20260809 16:23:40.055051 14314 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.51.148:34103
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292417722232-21710-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292417722232-21710-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.130.keytab
--principal=kudu/127.21.51.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.51.130:43191
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292417722232-21710-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.51.130
--webserver_port=36157
--tserver_master_addrs=127.21.51.190:44381
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14314
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.51.130
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292417722232-21710-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:23:40.055333 14314 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:23:40.055573 14314 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:23:40.060582 14327 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:23:40.060604 14324 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:23:40.060599 14323 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:23:40.060883 14314 server_base.cc:1034] running on GCE node
I20260809 16:23:40.061093 14314 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:23:40.061342 14314 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:23:40.062503 14314 hybrid_clock.cc:648] HybridClock initialized: now 1786292620062490 us; error 31 us; skew 500 ppm
Aug 09 16:23:40 dist-test-slave-s11t krb5kdc[13156](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292620, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.51.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:23:40.065027 14314 init.cc:377] Logged in from keytab as kudu/127.21.51.130@KRBTEST.COM (short username kudu)
I20260809 16:23:40.065455 14314 webserver.cc:460] Webserver started at http://127.21.51.130:36157/ using document root <none> and password file <none>
I20260809 16:23:40.065704 14314 fs_manager.cc:359] Metadata directory not provided
I20260809 16:23:40.065791 14314 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:23:40.067135 14314 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:23:40.067719 14338 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:23:40.067896 14314 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 16:23:40.068001 14314 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292417722232-21710-0/minicluster-data/ts-1/data,/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292417722232-21710-0/minicluster-data/ts-1/wal
uuid: "108d298be817495d987fd45a969d96ba"
format_stamp: "Formatted at 2026-08-09 16:23:19 on dist-test-slave-s11t"
server_key: "61136fc1c27ac7e207ba3360818e7bf6"
server_key_iv: "344bc88d0dd8954136aaa8398445bb69"
server_key_version: "encryptionkey@0"
I20260809 16:23:40.068334 14314 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292417722232-21710-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292417722232-21710-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292417722232-21710-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:23:40.098351 14314 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:23:40.100356 14314 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:23:40.100528 14314 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:23:40.100805 14314 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:23:40.101130 14314 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:23:40.101184 14314 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:23:40.101246 14314 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:23:40.101279 14314 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:23:40.107172 14314 rpc_server.cc:307] RPC server started. Bound to: 127.21.51.130:43191
I20260809 16:23:40.107230 14511 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.51.130:43191 every 8 connection(s)
I20260809 16:23:40.107553 14314 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292417722232-21710-0/minicluster-data/ts-1/data/info.pb
Aug 09 16:23:40 dist-test-slave-s11t krb5kdc[13156](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292620, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.51.130@KRBTEST.COM for kudu/127.21.51.190@KRBTEST.COM
I20260809 16:23:40.116918 21710 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrNqUCn/build/release/bin/kudu as pid 14314
I20260809 16:23:40.116959 14310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:23:40.108252 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.51.130:45117 (local address 127.21.51.190:44381)
0809 16:23:40.108341 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0809 16:23:40.108344 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 16:23:40.108540 (+ 196us) server_negotiation.cc:407] Connection header received
0809 16:23:40.109097 (+ 557us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:23:40.109098 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:23:40.109108 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:23:40.109161 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:23:40.110089 (+ 928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:23:40.110725 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:23:40.112001 (+ 1276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:23:40.112199 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:23:40.112255 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:23:40.114390 (+ 2135us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:23:40.114391 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:23:40.114391 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:23:40.114398 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:23:40.115624 (+ 1226us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:23:40.115988 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:23:40.115989 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:23:40.115989 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:23:40.116040 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:23:40.116307 (+ 267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:23:40.116308 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:23:40.116308 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:23:40.116403 (+ 95us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:23:40.116453 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0809 16:23:40.116679 (+ 226us) server_negotiation.cc:299] Negotiation successful
0809 16:23:40.116820 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260809 16:23:40.117069 14514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:23:40.108226 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:44381 (local address 127.21.51.130:45117)
0809 16:23:40.108420 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0809 16:23:40.108455 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0809 16:23:40.109004 (+ 549us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:23:40.109230 (+ 226us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:23:40.109233 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:23:40.109815 (+ 582us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:23:40.110005 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:23:40.110009 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:23:40.110971 (+ 962us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:23:40.110972 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:23:40.111916 (+ 944us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:23:40.111918 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:23:40.112278 (+ 360us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:23:40.112279 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:23:40.112379 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:23:40.112897 (+ 518us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:23:40.112905 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:23:40.114303 (+ 1398us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:23:40.115701 (+ 1398us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:23:40.115706 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:23:40.115708 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:23:40.115921 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:23:40.116123 (+ 202us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:23:40.116124 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:23:40.116124 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:23:40.116242 (+ 118us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:23:40.116461 (+ 219us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:23:40.116465 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:23:40.116571 (+ 106us) client_negotiation.cc:769] Sending connection context
0809 16:23:40.116652 (+ 81us) client_negotiation.cc:240] Negotiation successful
0809 16:23:40.116865 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":116,"threads_started":1}
I20260809 16:23:40.117246 21710 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrNqUCn/build/release/bin/kudu
/tmp/dist-test-taskrNqUCn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292417722232-21710-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292417722232-21710-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292417722232-21710-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292417722232-21710-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.51.131:40517
--local_ip_for_outbound_sockets=127.21.51.131
--tserver_master_addrs=127.21.51.190:44381
--webserver_port=37545
--webserver_interface=127.21.51.131
--keytab_file=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.131.keytab
--principal=kudu/127.21.51.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.51.148:34103
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292417722232-21710-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292417722232-21710-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292417722232-21710-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 16:23:40.117642 14512 heartbeater.cc:347] Connected to a master server at 127.21.51.190:44381
I20260809 16:23:40.117750 14512 heartbeater.cc:464] Registering TS with master...
I20260809 16:23:40.117926 14512 heartbeater.cc:511] Master 127.21.51.190:44381 requested a full tablet report, sending...
I20260809 16:23:40.118371 14032 ts_manager.cc:194] Registered new tserver with Master: 108d298be817495d987fd45a969d96ba (127.21.51.130:43191)
I20260809 16:23:40.118925 14032 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.51.130@KRBTEST.COM'} at 127.21.51.130:45117
W20260809 16:23:40.200852 14515 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:23:40.201046 14515 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:23:40.201071 14515 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:23:40.202193 14515 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:23:40.202236 14515 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:23:40.202262 14515 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.51.131
I20260809 16:23:40.203797 14515 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.51.148:34103
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292417722232-21710-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292417722232-21710-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.131.keytab
--principal=kudu/127.21.51.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.51.131:40517
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292417722232-21710-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.51.131
--webserver_port=37545
--tserver_master_addrs=127.21.51.190:44381
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14515
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.51.131
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292417722232-21710-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:23:40.204047 14515 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:23:40.204283 14515 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:23:40.209427 14524 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:23:40.209502 14521 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:23:40.209491 14515 server_base.cc:1034] running on GCE node
W20260809 16:23:40.209461 14520 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:23:40.209901 14515 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:23:40.210158 14515 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:23:40.211329 14515 hybrid_clock.cc:648] HybridClock initialized: now 1786292620211322 us; error 35 us; skew 500 ppm
Aug 09 16:23:40 dist-test-slave-s11t krb5kdc[13156](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292620, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.51.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:23:40.214010 14515 init.cc:377] Logged in from keytab as kudu/127.21.51.131@KRBTEST.COM (short username kudu)
I20260809 16:23:40.214429 14515 webserver.cc:460] Webserver started at http://127.21.51.131:37545/ using document root <none> and password file <none>
I20260809 16:23:40.214674 14515 fs_manager.cc:359] Metadata directory not provided
I20260809 16:23:40.214777 14515 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:23:40.216135 14515 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:23:40.216794 14536 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:23:40.216969 14515 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:23:40.217072 14515 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292417722232-21710-0/minicluster-data/ts-2/data,/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292417722232-21710-0/minicluster-data/ts-2/wal
uuid: "6aa3f90a2d2042e08594f1dd05d7fe30"
format_stamp: "Formatted at 2026-08-09 16:23:19 on dist-test-slave-s11t"
server_key: "3002adbb5d94446f03cb87eaed035f30"
server_key_iv: "e451942046f83051fdc9d91823d2a973"
server_key_version: "encryptionkey@0"
I20260809 16:23:40.217458 14515 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292417722232-21710-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292417722232-21710-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292417722232-21710-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:23:40.236618 14515 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:23:40.238960 14515 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:23:40.239128 14515 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:23:40.239362 14515 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:23:40.239663 14515 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:23:40.239717 14515 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:23:40.239769 14515 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:23:40.239797 14515 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:23:40.246253 14515 rpc_server.cc:307] RPC server started. Bound to: 127.21.51.131:40517
I20260809 16:23:40.246385 14711 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.51.131:40517 every 8 connection(s)
I20260809 16:23:40.246727 14515 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292417722232-21710-0/minicluster-data/ts-2/data/info.pb
I20260809 16:23:40.253667 21710 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrNqUCn/build/release/bin/kudu as pid 14515
Aug 09 16:23:40 dist-test-slave-s11t krb5kdc[13156](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292620, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.51.131@KRBTEST.COM for kudu/127.21.51.190@KRBTEST.COM
I20260809 16:23:40.258275 14310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:23:40.247687 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.51.131:38233 (local address 127.21.51.190:44381)
0809 16:23:40.247740 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0809 16:23:40.247742 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:23:40.247999 (+ 257us) server_negotiation.cc:407] Connection header received
0809 16:23:40.248533 (+ 534us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:23:40.248535 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:23:40.248548 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:23:40.248612 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:23:40.250074 (+ 1462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:23:40.250847 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:23:40.252501 (+ 1654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:23:40.252845 (+ 344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:23:40.252916 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:23:40.255407 (+ 2491us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:23:40.255409 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:23:40.255409 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:23:40.255417 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:23:40.256803 (+ 1386us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:23:40.257286 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:23:40.257287 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:23:40.257287 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:23:40.257350 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:23:40.257625 (+ 275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:23:40.257626 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:23:40.257626 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:23:40.257739 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:23:40.257808 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0809 16:23:40.258091 (+ 283us) server_negotiation.cc:299] Negotiation successful
0809 16:23:40.258182 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260809 16:23:40.258474 14716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:23:40.247667 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:44381 (local address 127.21.51.131:38233)
0809 16:23:40.247863 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0809 16:23:40.247925 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0809 16:23:40.248420 (+ 495us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:23:40.248710 (+ 290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:23:40.248714 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:23:40.249559 (+ 845us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:23:40.249887 (+ 328us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:23:40.249893 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:23:40.250956 (+ 1063us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:23:40.250957 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:23:40.252404 (+ 1447us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:23:40.252407 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:23:40.252955 (+ 548us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:23:40.252956 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:23:40.253107 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:23:40.253667 (+ 560us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:23:40.253677 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:23:40.255289 (+ 1612us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:23:40.256888 (+ 1599us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:23:40.256895 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:23:40.256897 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:23:40.257195 (+ 298us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:23:40.257433 (+ 238us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:23:40.257434 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:23:40.257434 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:23:40.257546 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:23:40.257811 (+ 265us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:23:40.257818 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:23:40.257960 (+ 142us) client_negotiation.cc:769] Sending connection context
0809 16:23:40.258068 (+ 108us) client_negotiation.cc:240] Negotiation successful
0809 16:23:40.258228 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":100,"threads_started":1}
I20260809 16:23:40.259271 14713 heartbeater.cc:347] Connected to a master server at 127.21.51.190:44381
I20260809 16:23:40.259395 14713 heartbeater.cc:464] Registering TS with master...
I20260809 16:23:40.259596 14713 heartbeater.cc:511] Master 127.21.51.190:44381 requested a full tablet report, sending...
I20260809 16:23:40.260031 14032 ts_manager.cc:194] Registered new tserver with Master: 6aa3f90a2d2042e08594f1dd05d7fe30 (127.21.51.131:40517)
I20260809 16:23:40.260406 14032 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.51.131@KRBTEST.COM'} at 127.21.51.131:38233
I20260809 16:23:40.261685 14718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:23:40.254158 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59118 (local address 127.21.51.190:44381)
0809 16:23:40.254299 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0809 16:23:40.254302 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 16:23:40.254433 (+ 131us) server_negotiation.cc:407] Connection header received
0809 16:23:40.254515 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:23:40.254516 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:23:40.254526 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:23:40.254600 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:23:40.255268 (+ 668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:23:40.255881 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:23:40.256824 (+ 943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:23:40.257040 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:23:40.257116 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:23:40.258330 (+ 1214us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:23:40.258331 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:23:40.258332 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:23:40.258339 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:23:40.260149 (+ 1810us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:23:40.260497 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:23:40.260498 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:23:40.260498 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:23:40.260564 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:23:40.261169 (+ 605us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:23:40.261169 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:23:40.261170 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:23:40.261275 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:23:40.261465 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0809 16:23:40.261503 (+ 38us) server_negotiation.cc:299] Negotiation successful
0809 16:23:40.261585 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":119,"thread_start_us":96,"threads_started":1}
I20260809 16:23:40.262485 21710 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 16:23:40.264937 14718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:23:40.262806 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59130 (local address 127.21.51.190:44381)
0809 16:23:40.262841 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0809 16:23:40.262842 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 16:23:40.262956 (+ 114us) server_negotiation.cc:407] Connection header received
0809 16:23:40.262999 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:23:40.263000 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:23:40.263026 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:23:40.263082 (+ 56us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 16:23:40.263177 (+ 95us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:23:40.263612 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:23:40.264303 (+ 691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:23:40.264482 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:23:40.264534 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:23:40.264665 (+ 131us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 16:23:40.264752 (+ 87us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0809 16:23:40.264898 (+ 146us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:59130: token expired
Metrics: {"server-negotiator.queue_time_us":16}
W20260809 16:23:40.264978 14718 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:59130: token expired
W20260809 16:23:40.265246 13923 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.21.51.190:44381)
I20260809 16:23:40.270761 14718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:23:40.265493 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59142 (local address 127.21.51.190:44381)
0809 16:23:40.265528 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0809 16:23:40.265529 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 16:23:40.265569 (+ 40us) server_negotiation.cc:407] Connection header received
0809 16:23:40.265637 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:23:40.265638 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:23:40.265645 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:23:40.265717 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:23:40.266146 (+ 429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:23:40.266568 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:23:40.267197 (+ 629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:23:40.267367 (+ 170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:23:40.267415 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:23:40.268425 (+ 1010us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:23:40.268426 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:23:40.268426 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:23:40.268432 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:23:40.269948 (+ 1516us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:23:40.270230 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:23:40.270230 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:23:40.270231 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:23:40.270281 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:23:40.270425 (+ 144us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:23:40.270426 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:23:40.270426 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:23:40.270506 (+ 80us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:23:40.270572 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0809 16:23:40.270653 (+ 81us) server_negotiation.cc:299] Negotiation successful
0809 16:23:40.270707 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260809 16:23:40.271920 14032 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59142:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 16:23:40.273553 14032 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 16:23:40.275789 14204 tablet_service.cc:1467] Processing CreateTablet for tablet 8200527efad0406d986bb696bb3394f8 (DEFAULT_TABLE table=test-table [id=8c954901a15e42409c7e5d3ecd0e8fec]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:23:40.276002 14204 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8200527efad0406d986bb696bb3394f8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:23:40.277546 14140 tablet_bootstrap.cc:492] T 8200527efad0406d986bb696bb3394f8 P 7d67bdf64e544c339ca52e32fcd4a5a7: Bootstrap starting.
I20260809 16:23:40.278033 14140 tablet_bootstrap.cc:654] T 8200527efad0406d986bb696bb3394f8 P 7d67bdf64e544c339ca52e32fcd4a5a7: Neither blocks nor log segments found. Creating new log.
I20260809 16:23:40.279050 14140 tablet_bootstrap.cc:492] T 8200527efad0406d986bb696bb3394f8 P 7d67bdf64e544c339ca52e32fcd4a5a7: No bootstrap required, opened a new log
I20260809 16:23:40.279106 14140 ts_tablet_manager.cc:1397] T 8200527efad0406d986bb696bb3394f8 P 7d67bdf64e544c339ca52e32fcd4a5a7: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 16:23:40.279260 14140 raft_consensus.cc:348] T 8200527efad0406d986bb696bb3394f8 P 7d67bdf64e544c339ca52e32fcd4a5a7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "108d298be817495d987fd45a969d96ba" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 43191 } } peers { permanent_uuid: "6aa3f90a2d2042e08594f1dd05d7fe30" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 40517 } } peers { permanent_uuid: "7d67bdf64e544c339ca52e32fcd4a5a7" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 37335 } }
I20260809 16:23:40.279326 14140 raft_consensus.cc:374] T 8200527efad0406d986bb696bb3394f8 P 7d67bdf64e544c339ca52e32fcd4a5a7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:23:40.279369 14140 raft_consensus.cc:729] T 8200527efad0406d986bb696bb3394f8 P 7d67bdf64e544c339ca52e32fcd4a5a7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7d67bdf64e544c339ca52e32fcd4a5a7, State: Initialized, Role: FOLLOWER
I20260809 16:23:40.279392 14721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:23:40.275493 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48788 (local address 127.21.51.130:43191)
0809 16:23:40.275664 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0809 16:23:40.275669 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 16:23:40.275679 (+ 10us) server_negotiation.cc:407] Connection header received
0809 16:23:40.275725 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:23:40.275728 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:23:40.275745 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:23:40.275826 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:23:40.275930 (+ 104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:23:40.276962 (+ 1032us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:23:40.278183 (+ 1221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:23:40.279013 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:23:40.279099 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:23:40.279160 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0809 16:23:40.279265 (+ 105us) server_negotiation.cc:299] Negotiation successful
0809 16:23:40.279307 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135,"thread_start_us":101,"threads_started":1}
I20260809 16:23:40.279417 14140 consensus_queue.cc:260] T 8200527efad0406d986bb696bb3394f8 P 7d67bdf64e544c339ca52e32fcd4a5a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "108d298be817495d987fd45a969d96ba" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 43191 } } peers { permanent_uuid: "6aa3f90a2d2042e08594f1dd05d7fe30" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 40517 } } peers { permanent_uuid: "7d67bdf64e544c339ca52e32fcd4a5a7" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 37335 } }
I20260809 16:23:40.279441 14315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:23:40.275394 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.130:43191 (local address 127.0.0.1:48788)
0809 16:23:40.275440 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0809 16:23:40.275450 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 16:23:40.275487 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:23:40.275827 (+ 340us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:23:40.275827 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:23:40.275828 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:23:40.275872 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:23:40.275874 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:23:40.277052 (+ 1178us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:23:40.277053 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:23:40.278096 (+ 1043us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:23:40.278097 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:23:40.279099 (+ 1002us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:23:40.279100 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:23:40.279171 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:23:40.279192 (+ 21us) client_negotiation.cc:769] Sending connection context
0809 16:23:40.279251 (+ 59us) client_negotiation.cc:240] Negotiation successful
0809 16:23:40.279352 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32,"mutex_wait_us":45}
I20260809 16:23:40.279507 14140 ts_tablet_manager.cc:1428] T 8200527efad0406d986bb696bb3394f8 P 7d67bdf64e544c339ca52e32fcd4a5a7: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 16:23:40.280288 14408 tablet_service.cc:1467] Processing CreateTablet for tablet 8200527efad0406d986bb696bb3394f8 (DEFAULT_TABLE table=test-table [id=8c954901a15e42409c7e5d3ecd0e8fec]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:23:40.280671 14408 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8200527efad0406d986bb696bb3394f8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:23:40.281343 14720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:23:40.275427 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.131:40517 (local address 127.0.0.1:48334)
0809 16:23:40.275678 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0809 16:23:40.275688 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 16:23:40.275735 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:23:40.277840 (+ 2105us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:23:40.277842 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:23:40.277844 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:23:40.277879 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:23:40.277881 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:23:40.279155 (+ 1274us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:23:40.279157 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:23:40.280309 (+ 1152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:23:40.280312 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:23:40.281107 (+ 795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:23:40.281108 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:23:40.281172 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:23:40.281177 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 16:23:40.281237 (+ 60us) client_negotiation.cc:240] Negotiation successful
0809 16:23:40.281275 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":238,"thread_start_us":57,"threads_started":1}
I20260809 16:23:40.281364 14722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:23:40.275605 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48334 (local address 127.21.51.131:40517)
0809 16:23:40.277646 (+ 2041us) server_negotiation.cc:206] Beginning negotiation
0809 16:23:40.277651 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 16:23:40.277663 (+ 12us) server_negotiation.cc:407] Connection header received
0809 16:23:40.277705 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:23:40.277708 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:23:40.277727 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:23:40.277828 (+ 101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:23:40.277949 (+ 121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:23:40.279051 (+ 1102us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:23:40.280385 (+ 1334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:23:40.281019 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:23:40.281105 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:23:40.281159 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0809 16:23:40.281250 (+ 91us) server_negotiation.cc:299] Negotiation successful
0809 16:23:40.281288 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2004,"thread_start_us":91,"threads_started":1}
I20260809 16:23:40.281926 14614 tablet_service.cc:1467] Processing CreateTablet for tablet 8200527efad0406d986bb696bb3394f8 (DEFAULT_TABLE table=test-table [id=8c954901a15e42409c7e5d3ecd0e8fec]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:23:40.282212 14614 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8200527efad0406d986bb696bb3394f8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:23:40.283595 14725 tablet_bootstrap.cc:492] T 8200527efad0406d986bb696bb3394f8 P 108d298be817495d987fd45a969d96ba: Bootstrap starting.
I20260809 16:23:40.284289 14725 tablet_bootstrap.cc:654] T 8200527efad0406d986bb696bb3394f8 P 108d298be817495d987fd45a969d96ba: Neither blocks nor log segments found. Creating new log.
I20260809 16:23:40.284965 14725 log.cc:824] T 8200527efad0406d986bb696bb3394f8 P 108d298be817495d987fd45a969d96ba: Log is configured to *not* fsync() on all Append() calls
I20260809 16:23:40.285373 14732 tablet_bootstrap.cc:492] T 8200527efad0406d986bb696bb3394f8 P 6aa3f90a2d2042e08594f1dd05d7fe30: Bootstrap starting.
I20260809 16:23:40.285738 14725 tablet_bootstrap.cc:492] T 8200527efad0406d986bb696bb3394f8 P 108d298be817495d987fd45a969d96ba: No bootstrap required, opened a new log
I20260809 16:23:40.285796 14725 ts_tablet_manager.cc:1397] T 8200527efad0406d986bb696bb3394f8 P 108d298be817495d987fd45a969d96ba: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 16:23:40.286005 14732 tablet_bootstrap.cc:654] T 8200527efad0406d986bb696bb3394f8 P 6aa3f90a2d2042e08594f1dd05d7fe30: Neither blocks nor log segments found. Creating new log.
I20260809 16:23:40.286487 14732 log.cc:824] T 8200527efad0406d986bb696bb3394f8 P 6aa3f90a2d2042e08594f1dd05d7fe30: Log is configured to *not* fsync() on all Append() calls
I20260809 16:23:40.287302 14732 tablet_bootstrap.cc:492] T 8200527efad0406d986bb696bb3394f8 P 6aa3f90a2d2042e08594f1dd05d7fe30: No bootstrap required, opened a new log
I20260809 16:23:40.287379 14732 ts_tablet_manager.cc:1397] T 8200527efad0406d986bb696bb3394f8 P 6aa3f90a2d2042e08594f1dd05d7fe30: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 16:23:40.287492 14725 raft_consensus.cc:348] T 8200527efad0406d986bb696bb3394f8 P 108d298be817495d987fd45a969d96ba [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "108d298be817495d987fd45a969d96ba" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 43191 } } peers { permanent_uuid: "6aa3f90a2d2042e08594f1dd05d7fe30" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 40517 } } peers { permanent_uuid: "7d67bdf64e544c339ca52e32fcd4a5a7" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 37335 } }
I20260809 16:23:40.287639 14725 raft_consensus.cc:374] T 8200527efad0406d986bb696bb3394f8 P 108d298be817495d987fd45a969d96ba [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:23:40.287662 14725 raft_consensus.cc:729] T 8200527efad0406d986bb696bb3394f8 P 108d298be817495d987fd45a969d96ba [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 108d298be817495d987fd45a969d96ba, State: Initialized, Role: FOLLOWER
I20260809 16:23:40.287791 14725 consensus_queue.cc:260] T 8200527efad0406d986bb696bb3394f8 P 108d298be817495d987fd45a969d96ba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "108d298be817495d987fd45a969d96ba" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 43191 } } peers { permanent_uuid: "6aa3f90a2d2042e08594f1dd05d7fe30" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 40517 } } peers { permanent_uuid: "7d67bdf64e544c339ca52e32fcd4a5a7" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 37335 } }
I20260809 16:23:40.288061 14725 ts_tablet_manager.cc:1428] T 8200527efad0406d986bb696bb3394f8 P 108d298be817495d987fd45a969d96ba: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260809 16:23:40.288125 14512 heartbeater.cc:503] Master 127.21.51.190:44381 was elected leader, sending a full tablet report...
I20260809 16:23:40.289320 14732 raft_consensus.cc:348] T 8200527efad0406d986bb696bb3394f8 P 6aa3f90a2d2042e08594f1dd05d7fe30 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "108d298be817495d987fd45a969d96ba" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 43191 } } peers { permanent_uuid: "6aa3f90a2d2042e08594f1dd05d7fe30" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 40517 } } peers { permanent_uuid: "7d67bdf64e544c339ca52e32fcd4a5a7" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 37335 } }
I20260809 16:23:40.289475 14732 raft_consensus.cc:374] T 8200527efad0406d986bb696bb3394f8 P 6aa3f90a2d2042e08594f1dd05d7fe30 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:23:40.289521 14732 raft_consensus.cc:729] T 8200527efad0406d986bb696bb3394f8 P 6aa3f90a2d2042e08594f1dd05d7fe30 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6aa3f90a2d2042e08594f1dd05d7fe30, State: Initialized, Role: FOLLOWER
I20260809 16:23:40.289628 14732 consensus_queue.cc:260] T 8200527efad0406d986bb696bb3394f8 P 6aa3f90a2d2042e08594f1dd05d7fe30 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "108d298be817495d987fd45a969d96ba" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 43191 } } peers { permanent_uuid: "6aa3f90a2d2042e08594f1dd05d7fe30" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 40517 } } peers { permanent_uuid: "7d67bdf64e544c339ca52e32fcd4a5a7" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 37335 } }
I20260809 16:23:40.289909 14732 ts_tablet_manager.cc:1428] T 8200527efad0406d986bb696bb3394f8 P 6aa3f90a2d2042e08594f1dd05d7fe30: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260809 16:23:40.289922 14713 heartbeater.cc:503] Master 127.21.51.190:44381 was elected leader, sending a full tablet report...
W20260809 16:23:40.358115 14513 tablet.cc:2369] T 8200527efad0406d986bb696bb3394f8 P 108d298be817495d987fd45a969d96ba: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 16:23:40.444367 14737 raft_consensus.cc:482] T 8200527efad0406d986bb696bb3394f8 P 6aa3f90a2d2042e08594f1dd05d7fe30 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 16:23:40.444499 14737 raft_consensus.cc:504] T 8200527efad0406d986bb696bb3394f8 P 6aa3f90a2d2042e08594f1dd05d7fe30 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "108d298be817495d987fd45a969d96ba" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 43191 } } peers { permanent_uuid: "6aa3f90a2d2042e08594f1dd05d7fe30" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 40517 } } peers { permanent_uuid: "7d67bdf64e544c339ca52e32fcd4a5a7" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 37335 } }
I20260809 16:23:40.444897 14737 leader_election.cc:288] T 8200527efad0406d986bb696bb3394f8 P 6aa3f90a2d2042e08594f1dd05d7fe30 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 108d298be817495d987fd45a969d96ba (127.21.51.130:43191), 7d67bdf64e544c339ca52e32fcd4a5a7 (127.21.51.129:37335)
I20260809 16:23:40.448022 14716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:23:40.445090 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.130:43191 (local address 127.21.51.131:40803)
0809 16:23:40.445148 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0809 16:23:40.445161 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 16:23:40.445223 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:23:40.445402 (+ 179us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:23:40.445403 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:23:40.445405 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:23:40.445494 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:23:40.445496 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:23:40.446246 (+ 750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:23:40.446247 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:23:40.447182 (+ 935us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:23:40.447185 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:23:40.447766 (+ 581us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:23:40.447767 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:23:40.447846 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:23:40.447849 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 16:23:40.447905 (+ 56us) client_negotiation.cc:240] Negotiation successful
0809 16:23:40.447944 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37,"mutex_wait_us":2}
I20260809 16:23:40.448057 14721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:23:40.445123 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.51.131:40803 (local address 127.21.51.130:43191)
0809 16:23:40.445168 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0809 16:23:40.445170 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:23:40.445257 (+ 87us) server_negotiation.cc:407] Connection header received
0809 16:23:40.445298 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:23:40.445299 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:23:40.445311 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:23:40.445389 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:23:40.445598 (+ 209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:23:40.446162 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:23:40.447289 (+ 1127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:23:40.447678 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:23:40.447736 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:23:40.447744 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0809 16:23:40.447930 (+ 186us) server_negotiation.cc:299] Negotiation successful
0809 16:23:40.447983 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260809 16:23:40.448094 14317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:23:40.445173 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.51.131:43183 (local address 127.21.51.129:37335)
0809 16:23:40.445221 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0809 16:23:40.445223 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:23:40.445372 (+ 149us) server_negotiation.cc:407] Connection header received
0809 16:23:40.445410 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:23:40.445411 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:23:40.445439 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:23:40.445520 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:23:40.445753 (+ 233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:23:40.446277 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:23:40.447307 (+ 1030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:23:40.447736 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:23:40.447809 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:23:40.447895 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0809 16:23:40.447980 (+ 85us) server_negotiation.cc:299] Negotiation successful
0809 16:23:40.448019 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
W20260809 16:23:40.448182 14305 tablet.cc:2369] T 8200527efad0406d986bb696bb3394f8 P 7d67bdf64e544c339ca52e32fcd4a5a7: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 16:23:40.448506 14739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:23:40.445090 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.129:37335 (local address 127.21.51.131:43183)
0809 16:23:40.445266 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0809 16:23:40.445278 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 16:23:40.445347 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:23:40.445588 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:23:40.445589 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:23:40.445592 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:23:40.445668 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:23:40.445670 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:23:40.446382 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:23:40.446383 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:23:40.447182 (+ 799us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:23:40.447184 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:23:40.447827 (+ 643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:23:40.447828 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:23:40.447911 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:23:40.447916 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 16:23:40.447976 (+ 60us) client_negotiation.cc:240] Negotiation successful
0809 16:23:40.448013 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":155,"mutex_wait_us":13,"thread_start_us":99,"threads_started":1}
I20260809 16:23:40.448477 14436 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8200527efad0406d986bb696bb3394f8" candidate_uuid: "6aa3f90a2d2042e08594f1dd05d7fe30" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "108d298be817495d987fd45a969d96ba" is_pre_election: true
I20260809 16:23:40.448652 14436 raft_consensus.cc:2454] T 8200527efad0406d986bb696bb3394f8 P 108d298be817495d987fd45a969d96ba [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6aa3f90a2d2042e08594f1dd05d7fe30 in term 0.
I20260809 16:23:40.448887 14233 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8200527efad0406d986bb696bb3394f8" candidate_uuid: "6aa3f90a2d2042e08594f1dd05d7fe30" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7d67bdf64e544c339ca52e32fcd4a5a7" is_pre_election: true
I20260809 16:23:40.448972 14538 leader_election.cc:302] T 8200527efad0406d986bb696bb3394f8 P 6aa3f90a2d2042e08594f1dd05d7fe30 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 108d298be817495d987fd45a969d96ba, 6aa3f90a2d2042e08594f1dd05d7fe30; no voters:
I20260809 16:23:40.449055 14233 raft_consensus.cc:2454] T 8200527efad0406d986bb696bb3394f8 P 7d67bdf64e544c339ca52e32fcd4a5a7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6aa3f90a2d2042e08594f1dd05d7fe30 in term 0.
I20260809 16:23:40.449106 14737 raft_consensus.cc:2781] T 8200527efad0406d986bb696bb3394f8 P 6aa3f90a2d2042e08594f1dd05d7fe30 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 16:23:40.449165 14737 raft_consensus.cc:482] T 8200527efad0406d986bb696bb3394f8 P 6aa3f90a2d2042e08594f1dd05d7fe30 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 16:23:40.449191 14737 raft_consensus.cc:3037] T 8200527efad0406d986bb696bb3394f8 P 6aa3f90a2d2042e08594f1dd05d7fe30 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:23:40.449958 14737 raft_consensus.cc:504] T 8200527efad0406d986bb696bb3394f8 P 6aa3f90a2d2042e08594f1dd05d7fe30 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "108d298be817495d987fd45a969d96ba" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 43191 } } peers { permanent_uuid: "6aa3f90a2d2042e08594f1dd05d7fe30" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 40517 } } peers { permanent_uuid: "7d67bdf64e544c339ca52e32fcd4a5a7" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 37335 } }
I20260809 16:23:40.450093 14737 leader_election.cc:288] T 8200527efad0406d986bb696bb3394f8 P 6aa3f90a2d2042e08594f1dd05d7fe30 [CANDIDATE]: Term 1 election: Requested vote from peers 108d298be817495d987fd45a969d96ba (127.21.51.130:43191), 7d67bdf64e544c339ca52e32fcd4a5a7 (127.21.51.129:37335)
I20260809 16:23:40.450238 14436 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8200527efad0406d986bb696bb3394f8" candidate_uuid: "6aa3f90a2d2042e08594f1dd05d7fe30" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "108d298be817495d987fd45a969d96ba"
I20260809 16:23:40.450260 14233 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8200527efad0406d986bb696bb3394f8" candidate_uuid: "6aa3f90a2d2042e08594f1dd05d7fe30" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7d67bdf64e544c339ca52e32fcd4a5a7"
I20260809 16:23:40.450328 14233 raft_consensus.cc:3037] T 8200527efad0406d986bb696bb3394f8 P 7d67bdf64e544c339ca52e32fcd4a5a7 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:23:40.450328 14436 raft_consensus.cc:3037] T 8200527efad0406d986bb696bb3394f8 P 108d298be817495d987fd45a969d96ba [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:23:40.451256 14436 raft_consensus.cc:2454] T 8200527efad0406d986bb696bb3394f8 P 108d298be817495d987fd45a969d96ba [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6aa3f90a2d2042e08594f1dd05d7fe30 in term 1.
I20260809 16:23:40.451256 14233 raft_consensus.cc:2454] T 8200527efad0406d986bb696bb3394f8 P 7d67bdf64e544c339ca52e32fcd4a5a7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6aa3f90a2d2042e08594f1dd05d7fe30 in term 1.
I20260809 16:23:40.451581 14541 leader_election.cc:302] T 8200527efad0406d986bb696bb3394f8 P 6aa3f90a2d2042e08594f1dd05d7fe30 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6aa3f90a2d2042e08594f1dd05d7fe30, 7d67bdf64e544c339ca52e32fcd4a5a7; no voters:
I20260809 16:23:40.451699 14737 raft_consensus.cc:2781] T 8200527efad0406d986bb696bb3394f8 P 6aa3f90a2d2042e08594f1dd05d7fe30 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:23:40.451875 14737 raft_consensus.cc:686] T 8200527efad0406d986bb696bb3394f8 P 6aa3f90a2d2042e08594f1dd05d7fe30 [term 1 LEADER]: Becoming Leader. State: Replica: 6aa3f90a2d2042e08594f1dd05d7fe30, State: Running, Role: LEADER
I20260809 16:23:40.451995 14737 consensus_queue.cc:237] T 8200527efad0406d986bb696bb3394f8 P 6aa3f90a2d2042e08594f1dd05d7fe30 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "108d298be817495d987fd45a969d96ba" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 43191 } } peers { permanent_uuid: "6aa3f90a2d2042e08594f1dd05d7fe30" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 40517 } } peers { permanent_uuid: "7d67bdf64e544c339ca52e32fcd4a5a7" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 37335 } }
I20260809 16:23:40.452903 14032 catalog_manager.cc:5495] T 8200527efad0406d986bb696bb3394f8 P 6aa3f90a2d2042e08594f1dd05d7fe30 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6aa3f90a2d2042e08594f1dd05d7fe30 (127.21.51.131). New cstate: current_term: 1 leader_uuid: "6aa3f90a2d2042e08594f1dd05d7fe30" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "108d298be817495d987fd45a969d96ba" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 43191 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6aa3f90a2d2042e08594f1dd05d7fe30" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 40517 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7d67bdf64e544c339ca52e32fcd4a5a7" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 37335 } health_report { overall_health: UNKNOWN } } }
I20260809 16:23:40.459668 14722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:23:40.457500 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48342 (local address 127.21.51.131:40517)
0809 16:23:40.457561 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0809 16:23:40.457563 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:23:40.457573 (+ 10us) server_negotiation.cc:407] Connection header received
0809 16:23:40.457626 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:23:40.457627 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:23:40.457643 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:23:40.457697 (+ 54us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 16:23:40.457814 (+ 117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:23:40.458274 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:23:40.458983 (+ 709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:23:40.459164 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:23:40.459213 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:23:40.459347 (+ 134us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 16:23:40.459452 (+ 105us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 16:23:40.459515 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0809 16:23:40.459546 (+ 31us) server_negotiation.cc:299] Negotiation successful
0809 16:23:40.459565 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260809 16:23:40.461619 14436 raft_consensus.cc:1261] T 8200527efad0406d986bb696bb3394f8 P 108d298be817495d987fd45a969d96ba [term 1 FOLLOWER]: Refusing update from remote peer 6aa3f90a2d2042e08594f1dd05d7fe30: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 16:23:40.461971 14233 raft_consensus.cc:1261] T 8200527efad0406d986bb696bb3394f8 P 7d67bdf64e544c339ca52e32fcd4a5a7 [term 1 FOLLOWER]: Refusing update from remote peer 6aa3f90a2d2042e08594f1dd05d7fe30: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 16:23:40.462080 14737 consensus_queue.cc:1035] T 8200527efad0406d986bb696bb3394f8 P 6aa3f90a2d2042e08594f1dd05d7fe30 [LEADER]: Connected to new peer: Peer: permanent_uuid: "108d298be817495d987fd45a969d96ba" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 43191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 16:23:40.462276 14740 consensus_queue.cc:1035] T 8200527efad0406d986bb696bb3394f8 P 6aa3f90a2d2042e08594f1dd05d7fe30 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d67bdf64e544c339ca52e32fcd4a5a7" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 37335 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 16:23:40.465834 14747 mvcc.cc:205] Tried to move back new op lower bound from 7316654573408088064 to 7316654573372076032. Current Snapshot: MvccSnapshot[applied={T|T < 7316654573408088064}]
I20260809 16:23:40.465965 14749 mvcc.cc:205] Tried to move back new op lower bound from 7316654573408088064 to 7316654573372076032. Current Snapshot: MvccSnapshot[applied={T|T < 7316654573408088064 or (T in {7316654573408088064})}]
I20260809 16:23:40.468384 14031 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59142:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 16:23:40.468505 14031 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59142:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 16:23:40.469403 14031 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8c44acc0f3e8446693dc64d77163f6b4: Sending DeleteTablet for 3 replicas of tablet 8200527efad0406d986bb696bb3394f8
I20260809 16:23:40.469725 14614 tablet_service.cc:1514] Processing DeleteTablet for tablet 8200527efad0406d986bb696bb3394f8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:23:40 UTC) from {username='kudu', principal='kudu/127.21.51.190@KRBTEST.COM'} at 127.0.0.1:48334
I20260809 16:23:40.469777 14204 tablet_service.cc:1514] Processing DeleteTablet for tablet 8200527efad0406d986bb696bb3394f8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:23:40 UTC) from {username='kudu', principal='kudu/127.21.51.190@KRBTEST.COM'} at 127.0.0.1:59664
I20260809 16:23:40.469805 14408 tablet_service.cc:1514] Processing DeleteTablet for tablet 8200527efad0406d986bb696bb3394f8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:23:40 UTC) from {username='kudu', principal='kudu/127.21.51.190@KRBTEST.COM'} at 127.0.0.1:48788
I20260809 16:23:40.469919 14318 tablet_replica.cc:330] T 8200527efad0406d986bb696bb3394f8 P 7d67bdf64e544c339ca52e32fcd4a5a7: stopping tablet replica
I20260809 16:23:40.469982 14318 raft_consensus.cc:2229] T 8200527efad0406d986bb696bb3394f8 P 7d67bdf64e544c339ca52e32fcd4a5a7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 16:23:40.470031 14318 raft_consensus.cc:2258] T 8200527efad0406d986bb696bb3394f8 P 7d67bdf64e544c339ca52e32fcd4a5a7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 16:23:40.470147 21710 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrNqUCn/build/release/bin/kudu with pid 14085
I20260809 16:23:40.470170 14759 tablet_replica.cc:330] T 8200527efad0406d986bb696bb3394f8 P 6aa3f90a2d2042e08594f1dd05d7fe30: stopping tablet replica
I20260809 16:23:40.477784 14759 raft_consensus.cc:2229] T 8200527efad0406d986bb696bb3394f8 P 6aa3f90a2d2042e08594f1dd05d7fe30 [term 1 LEADER]: Raft consensus shutting down.
I20260809 16:23:40.477986 14759 raft_consensus.cc:2258] T 8200527efad0406d986bb696bb3394f8 P 6aa3f90a2d2042e08594f1dd05d7fe30 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 16:23:40.478523 14759 ts_tablet_manager.cc:1905] T 8200527efad0406d986bb696bb3394f8 P 6aa3f90a2d2042e08594f1dd05d7fe30: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 16:23:40.479555 14762 tablet_replica.cc:330] T 8200527efad0406d986bb696bb3394f8 P 108d298be817495d987fd45a969d96ba: stopping tablet replica
I20260809 16:23:40.479725 14762 raft_consensus.cc:2229] T 8200527efad0406d986bb696bb3394f8 P 108d298be817495d987fd45a969d96ba [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 16:23:40.479930 14762 raft_consensus.cc:2258] T 8200527efad0406d986bb696bb3394f8 P 108d298be817495d987fd45a969d96ba [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 16:23:40.480520 14762 ts_tablet_manager.cc:1905] T 8200527efad0406d986bb696bb3394f8 P 108d298be817495d987fd45a969d96ba: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 16:23:40.480576 14759 ts_tablet_manager.cc:1918] T 8200527efad0406d986bb696bb3394f8 P 6aa3f90a2d2042e08594f1dd05d7fe30: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 16:23:40.480705 14759 log.cc:1196] T 8200527efad0406d986bb696bb3394f8 P 6aa3f90a2d2042e08594f1dd05d7fe30: Deleting WAL directory at /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292417722232-21710-0/minicluster-data/ts-2/wal/wals/8200527efad0406d986bb696bb3394f8
I20260809 16:23:40.481113 14759 ts_tablet_manager.cc:1939] T 8200527efad0406d986bb696bb3394f8 P 6aa3f90a2d2042e08594f1dd05d7fe30: Deleting consensus metadata
I20260809 16:23:40.481597 14012 catalog_manager.cc:4841] TS 6aa3f90a2d2042e08594f1dd05d7fe30 (127.21.51.131:40517): tablet 8200527efad0406d986bb696bb3394f8 (table test-table [id=8c954901a15e42409c7e5d3ecd0e8fec]) successfully deleted
I20260809 16:23:40.482589 14762 ts_tablet_manager.cc:1918] T 8200527efad0406d986bb696bb3394f8 P 108d298be817495d987fd45a969d96ba: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 16:23:40.482676 14762 log.cc:1196] T 8200527efad0406d986bb696bb3394f8 P 108d298be817495d987fd45a969d96ba: Deleting WAL directory at /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292417722232-21710-0/minicluster-data/ts-1/wal/wals/8200527efad0406d986bb696bb3394f8
I20260809 16:23:40.483021 14762 ts_tablet_manager.cc:1939] T 8200527efad0406d986bb696bb3394f8 P 108d298be817495d987fd45a969d96ba: Deleting consensus metadata
I20260809 16:23:40.483476 14011 catalog_manager.cc:4841] TS 108d298be817495d987fd45a969d96ba (127.21.51.130:43191): tablet 8200527efad0406d986bb696bb3394f8 (table test-table [id=8c954901a15e42409c7e5d3ecd0e8fec]) successfully deleted
W20260809 16:23:40.486078 14012 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.21.51.129:37335 (error 108)
I20260809 16:23:40.486675 14720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:23:40.486522 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.129:37335 (local address 127.0.0.1:59668)
0809 16:23:40.486587 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0809 16:23:40.486632 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.51.129:37335: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260809 16:23:40.486861 21710 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrNqUCn/build/release/bin/kudu with pid 14314
W20260809 16:23:40.486809 14012 catalog_manager.cc:4584] TS 7d67bdf64e544c339ca52e32fcd4a5a7 (127.21.51.129:37335): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 8200527efad0406d986bb696bb3394f8: Network error: Client connection negotiation failed: client connection to 127.21.51.129:37335: connect: Connection refused (error 111)
I20260809 16:23:40.504045 21710 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrNqUCn/build/release/bin/kudu with pid 14515
I20260809 16:23:40.516316 21710 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrNqUCn/build/release/bin/kudu with pid 13978
I20260809 16:24:00.525496 21710 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrNqUCn/build/release/bin/kudu
/tmp/dist-test-taskrNqUCn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292417722232-21710-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292417722232-21710-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292417722232-21710-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292417722232-21710-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.51.190:44381
--webserver_interface=127.21.51.190
--webserver_port=34771
--keytab_file=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.190.keytab
--principal=kudu/127.21.51.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.51.148:34103
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.51.190:44381
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292417722232-21710-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292417722232-21710-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292417722232-21710-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:24:00.608078 14765 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:24:00.608274 14765 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:24:00.608295 14765 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:24:00.609418 14765 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260809 16:24:00.609457 14765 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260809 16:24:00.609470 14765 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260809 16:24:00.609483 14765 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 16:24:00.609495 14765 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:24:00.609508 14765 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:24:00.609520 14765 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 16:24:00.609532 14765 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 16:24:00.610941 14765 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.51.148:34103
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292417722232-21710-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292417722232-21710-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.21.51.190:44381
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.190.keytab
--principal=kudu/127.21.51.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.51.190:44381
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292417722232-21710-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.51.190
--webserver_port=34771
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14765
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292417722232-21710-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:24:00.611136 14765 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:24:00.611307 14765 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 16:24:00.616464 14765 server_base.cc:1034] running on GCE node
W20260809 16:24:00.616518 14771 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:24:00.616437 14777 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:24:00.616438 14772 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:24:00.616933 14765 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:24:00.617247 14765 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:24:00.618417 14765 hybrid_clock.cc:648] HybridClock initialized: now 1786292640618405 us; error 46 us; skew 500 ppm
Aug 09 16:24:00 dist-test-slave-s11t krb5kdc[13156](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292640, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.51.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:24:00.620993 14765 init.cc:377] Logged in from keytab as kudu/127.21.51.190@KRBTEST.COM (short username kudu)
I20260809 16:24:00.621433 14765 webserver.cc:460] Webserver started at http://127.21.51.190:34771/ using document root <none> and password file <none>
I20260809 16:24:00.621677 14765 fs_manager.cc:359] Metadata directory not provided
I20260809 16:24:00.621745 14765 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:24:00.623075 14765 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260809 16:24:00.625018 14786 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:24:00.625201 14765 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260809 16:24:00.625306 14765 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292417722232-21710-0/minicluster-data/master-0/data,/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292417722232-21710-0/minicluster-data/master-0/wal
uuid: "8c44acc0f3e8446693dc64d77163f6b4"
format_stamp: "Formatted at 2026-08-09 16:23:18 on dist-test-slave-s11t"
server_key: "feee4944f16d07b04e52e984838a350c"
server_key_iv: "b5fdc51511f5019e4a395d2478e913f1"
server_key_version: "encryptionkey@0"
I20260809 16:24:00.625686 14765 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292417722232-21710-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292417722232-21710-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292417722232-21710-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:24:00.646095 14765 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:24:00.648082 14765 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:24:00.648257 14765 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:24:00.652498 14765 rpc_server.cc:307] RPC server started. Bound to: 127.21.51.190:44381
I20260809 16:24:00.652541 14862 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.51.190:44381 every 8 connection(s)
I20260809 16:24:00.652959 14765 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292417722232-21710-0/minicluster-data/master-0/data/info.pb
I20260809 16:24:00.654527 14863 sys_catalog.cc:263] Verifying existing consensus state
I20260809 16:24:00.655256 14863 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8c44acc0f3e8446693dc64d77163f6b4: Bootstrap starting.
I20260809 16:24:00.661420 21710 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrNqUCn/build/release/bin/kudu as pid 14765
I20260809 16:24:00.661772 21710 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrNqUCn/build/release/bin/kudu
/tmp/dist-test-taskrNqUCn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292417722232-21710-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292417722232-21710-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292417722232-21710-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292417722232-21710-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.51.129:37335
--local_ip_for_outbound_sockets=127.21.51.129
--tserver_master_addrs=127.21.51.190:44381
--webserver_port=40013
--webserver_interface=127.21.51.129
--keytab_file=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.129.keytab
--principal=kudu/127.21.51.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.51.148:34103
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292417722232-21710-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292417722232-21710-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292417722232-21710-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 16:24:00.664819 14863 log.cc:824] T 00000000000000000000000000000000 P 8c44acc0f3e8446693dc64d77163f6b4: Log is configured to *not* fsync() on all Append() calls
I20260809 16:24:00.667068 14863 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8c44acc0f3e8446693dc64d77163f6b4: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 16:24:00.667387 14863 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8c44acc0f3e8446693dc64d77163f6b4: Bootstrap complete.
I20260809 16:24:00.669740 14863 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8c44acc0f3e8446693dc64d77163f6b4 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c44acc0f3e8446693dc64d77163f6b4" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 44381 } }
I20260809 16:24:00.669972 14863 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8c44acc0f3e8446693dc64d77163f6b4 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c44acc0f3e8446693dc64d77163f6b4, State: Initialized, Role: FOLLOWER
I20260809 16:24:00.670137 14863 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8c44acc0f3e8446693dc64d77163f6b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c44acc0f3e8446693dc64d77163f6b4" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 44381 } }
I20260809 16:24:00.670217 14863 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8c44acc0f3e8446693dc64d77163f6b4 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 16:24:00.670272 14863 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8c44acc0f3e8446693dc64d77163f6b4 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 16:24:00.670341 14863 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8c44acc0f3e8446693dc64d77163f6b4 [term 2 FOLLOWER]: Advancing to term 3
I20260809 16:24:00.671342 14863 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8c44acc0f3e8446693dc64d77163f6b4 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c44acc0f3e8446693dc64d77163f6b4" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 44381 } }
I20260809 16:24:00.671459 14863 leader_election.cc:302] T 00000000000000000000000000000000 P 8c44acc0f3e8446693dc64d77163f6b4 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8c44acc0f3e8446693dc64d77163f6b4; no voters:
I20260809 16:24:00.671636 14863 leader_election.cc:288] T 00000000000000000000000000000000 P 8c44acc0f3e8446693dc64d77163f6b4 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260809 16:24:00.671705 14871 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8c44acc0f3e8446693dc64d77163f6b4 [term 3 FOLLOWER]: Leader election won for term 3
I20260809 16:24:00.671890 14871 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8c44acc0f3e8446693dc64d77163f6b4 [term 3 LEADER]: Becoming Leader. State: Replica: 8c44acc0f3e8446693dc64d77163f6b4, State: Running, Role: LEADER
I20260809 16:24:00.671908 14863 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8c44acc0f3e8446693dc64d77163f6b4 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 16:24:00.671989 14871 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8c44acc0f3e8446693dc64d77163f6b4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c44acc0f3e8446693dc64d77163f6b4" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 44381 } }
I20260809 16:24:00.672281 14871 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8c44acc0f3e8446693dc64d77163f6b4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8c44acc0f3e8446693dc64d77163f6b4. Latest consensus state: current_term: 3 leader_uuid: "8c44acc0f3e8446693dc64d77163f6b4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c44acc0f3e8446693dc64d77163f6b4" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 44381 } } }
I20260809 16:24:00.672384 14871 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8c44acc0f3e8446693dc64d77163f6b4 [sys.catalog]: This master's current role is: LEADER
I20260809 16:24:00.672286 14872 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8c44acc0f3e8446693dc64d77163f6b4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "8c44acc0f3e8446693dc64d77163f6b4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c44acc0f3e8446693dc64d77163f6b4" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 44381 } } }
I20260809 16:24:00.672583 14872 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8c44acc0f3e8446693dc64d77163f6b4 [sys.catalog]: This master's current role is: LEADER
I20260809 16:24:00.673080 14880 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 16:24:00.673923 14880 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 16:24:00.674026 14880 catalog_manager.cc:1259] Loaded cluster ID: ae92579412e5493faccfdf6bb3f0cb1f
I20260809 16:24:00.674048 14880 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 16:24:00.675287 14880 catalog_manager.cc:1504] Loading token signing keys...
I20260809 16:24:00.675484 14880 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 8c44acc0f3e8446693dc64d77163f6b4: Loaded TSK: 0
I20260809 16:24:00.675531 14880 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 8c44acc0f3e8446693dc64d77163f6b4: Loaded TSK: 1
F20260809 16:24:00.675755 14880 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7fa042f829ad google::LogMessage::Fail() at ??:0
@ 0x7fa042f8588a google::LogMessage::SendToLog() at ??:0
@ 0x7fa042f824de google::LogMessage::Flush() at ??:0
@ 0x7fa042f83a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7fa0456ef104 kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7fa0456ecd41 kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7fa04aee01cd kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7fa04aee048e kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7fa04aee0594 _ZNSt17_Function_handlerIFN4kudu6StatusEvEZNS0_6master14CatalogManager24PrepareForLeadershipTaskEvEUlvE3_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x7fa04aea6c48 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7fa04aeb0ad0 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7fa044e500d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7fa044e45954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7fa045ca96db start_thread at ??:0
@ 0x7fa041a8061f clone at ??:0
W20260809 16:24:00.782204 14867 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:24:00.782478 14867 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:24:00.782516 14867 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:24:00.784198 14867 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:24:00.784257 14867 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:24:00.784296 14867 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.51.129
I20260809 16:24:00.786516 14867 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.51.148:34103
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292417722232-21710-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292417722232-21710-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.129.keytab
--principal=kudu/127.21.51.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.51.129:37335
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292417722232-21710-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.51.129
--webserver_port=40013
--tserver_master_addrs=127.21.51.190:44381
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14867
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.51.129
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292417722232-21710-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:24:00.786782 14867 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:24:00.787070 14867 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:24:00.792549 14921 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:24:00.792557 14924 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:24:00.792712 14867 server_base.cc:1034] running on GCE node
W20260809 16:24:00.792558 14922 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:24:00.793085 14867 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:24:00.793371 14867 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:24:00.794562 14867 hybrid_clock.cc:648] HybridClock initialized: now 1786292640794540 us; error 44 us; skew 500 ppm
Aug 09 16:24:00 dist-test-slave-s11t krb5kdc[13156](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292640, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.51.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:24:00.797492 14867 init.cc:377] Logged in from keytab as kudu/127.21.51.129@KRBTEST.COM (short username kudu)
I20260809 16:24:00.797952 14867 webserver.cc:460] Webserver started at http://127.21.51.129:40013/ using document root <none> and password file <none>
I20260809 16:24:00.798174 14867 fs_manager.cc:359] Metadata directory not provided
I20260809 16:24:00.798259 14867 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:24:00.799643 14867 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260809 16:24:00.800395 14937 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:24:00.800588 14867 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 16:24:00.800709 14867 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292417722232-21710-0/minicluster-data/ts-0/data,/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292417722232-21710-0/minicluster-data/ts-0/wal
uuid: "7d67bdf64e544c339ca52e32fcd4a5a7"
format_stamp: "Formatted at 2026-08-09 16:23:19 on dist-test-slave-s11t"
server_key: "3899e01c0413848a2deae05fbcc2cec1"
server_key_iv: "6f19b55e88ae041ff9a866bc626841a2"
server_key_version: "encryptionkey@0"
I20260809 16:24:00.801072 14867 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292417722232-21710-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292417722232-21710-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292417722232-21710-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:24:00.809697 14867 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:24:00.811921 14867 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:24:00.812077 14867 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:24:00.812309 14867 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:24:00.812758 14945 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260809 16:24:00.814895 14867 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260809 16:24:00.814965 14867 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260809 16:24:00.815001 14867 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260809 16:24:00.815634 14867 ts_tablet_manager.cc:610] Registered 1 tablets
I20260809 16:24:00.815688 14867 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260809 16:24:00.815749 14945 tablet_bootstrap.cc:492] T 8200527efad0406d986bb696bb3394f8 P 7d67bdf64e544c339ca52e32fcd4a5a7: Bootstrap starting.
I20260809 16:24:00.822948 14867 rpc_server.cc:307] RPC server started. Bound to: 127.21.51.129:37335
I20260809 16:24:00.823068 15108 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.51.129:37335 every 8 connection(s)
I20260809 16:24:00.823377 14867 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292417722232-21710-0/minicluster-data/ts-0/data/info.pb
I20260809 16:24:00.826714 14945 log.cc:824] T 8200527efad0406d986bb696bb3394f8 P 7d67bdf64e544c339ca52e32fcd4a5a7: Log is configured to *not* fsync() on all Append() calls
I20260809 16:24:00.828162 21710 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrNqUCn/build/release/bin/kudu as pid 14867
I20260809 16:24:00.828469 21710 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrNqUCn/build/release/bin/kudu
/tmp/dist-test-taskrNqUCn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292417722232-21710-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292417722232-21710-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292417722232-21710-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292417722232-21710-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.51.130:43191
--local_ip_for_outbound_sockets=127.21.51.130
--tserver_master_addrs=127.21.51.190:44381
--webserver_port=36157
--webserver_interface=127.21.51.130
--keytab_file=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.130.keytab
--principal=kudu/127.21.51.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.51.148:34103
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292417722232-21710-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292417722232-21710-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292417722232-21710-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 16:24:00.829442 14945 tablet_bootstrap.cc:492] T 8200527efad0406d986bb696bb3394f8 P 7d67bdf64e544c339ca52e32fcd4a5a7: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 16:24:00.829771 14945 tablet_bootstrap.cc:492] T 8200527efad0406d986bb696bb3394f8 P 7d67bdf64e544c339ca52e32fcd4a5a7: Bootstrap complete.
I20260809 16:24:00.831053 14945 ts_tablet_manager.cc:1397] T 8200527efad0406d986bb696bb3394f8 P 7d67bdf64e544c339ca52e32fcd4a5a7: Time spent bootstrapping tablet: real 0.015s user 0.002s sys 0.011s
I20260809 16:24:00.832491 14945 raft_consensus.cc:348] T 8200527efad0406d986bb696bb3394f8 P 7d67bdf64e544c339ca52e32fcd4a5a7 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "108d298be817495d987fd45a969d96ba" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 43191 } } peers { permanent_uuid: "6aa3f90a2d2042e08594f1dd05d7fe30" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 40517 } } peers { permanent_uuid: "7d67bdf64e544c339ca52e32fcd4a5a7" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 37335 } }
I20260809 16:24:00.832878 14945 raft_consensus.cc:729] T 8200527efad0406d986bb696bb3394f8 P 7d67bdf64e544c339ca52e32fcd4a5a7 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7d67bdf64e544c339ca52e32fcd4a5a7, State: Initialized, Role: FOLLOWER
I20260809 16:24:00.833053 14945 consensus_queue.cc:260] T 8200527efad0406d986bb696bb3394f8 P 7d67bdf64e544c339ca52e32fcd4a5a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "108d298be817495d987fd45a969d96ba" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 43191 } } peers { permanent_uuid: "6aa3f90a2d2042e08594f1dd05d7fe30" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 40517 } } peers { permanent_uuid: "7d67bdf64e544c339ca52e32fcd4a5a7" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 37335 } }
I20260809 16:24:00.833374 14945 ts_tablet_manager.cc:1428] T 8200527efad0406d986bb696bb3394f8 P 7d67bdf64e544c339ca52e32fcd4a5a7: Time spent starting tablet: real 0.002s user 0.005s sys 0.000s
W20260809 16:24:00.905385 15113 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:24:00.824101 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:44381 (local address 127.21.51.129:38509)
0809 16:24:00.824266 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0809 16:24:00.824309 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0809 16:24:00.824847 (+ 538us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:24:00.905228 (+ 80381us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.51.190:44381: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":96,"threads_started":1}
W20260809 16:24:00.905654 15110 heartbeater.cc:644] Failed to heartbeat to 127.21.51.190:44381 (0 consecutive failures): Network error: Failed to ping master at 127.21.51.190:44381: Client connection negotiation failed: client connection to 127.21.51.190:44381: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260809 16:24:00.906009 15113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:00.905887 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:44381 (local address 127.21.51.129:60223)
0809 16:24:00.905944 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0809 16:24:00.905988 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.51.190:44381: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260809 16:24:00.906304 15113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:00.906217 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:44381 (local address 127.21.51.129:53227)
0809 16:24:00.906258 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0809 16:24:00.906282 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.51.190:44381: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
W20260809 16:24:00.906409 15110 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260809 16:24:00.917533 15117 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:24:00.917829 15117 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:24:00.917888 15117 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:24:00.919227 15117 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:24:00.919299 15117 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:24:00.919350 15117 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.51.130
I20260809 16:24:00.921066 15117 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.51.148:34103
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292417722232-21710-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292417722232-21710-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.130.keytab
--principal=kudu/127.21.51.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.51.130:43191
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292417722232-21710-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.51.130
--webserver_port=36157
--tserver_master_addrs=127.21.51.190:44381
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15117
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.51.130
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292417722232-21710-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:24:00.921350 15117 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:24:00.921602 15117 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:24:00.927279 15124 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:24:00.927289 15125 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:24:00.927366 15128 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:24:00.927570 15117 server_base.cc:1034] running on GCE node
I20260809 16:24:00.927798 15117 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:24:00.928037 15117 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:24:00.929179 15117 hybrid_clock.cc:648] HybridClock initialized: now 1786292640929159 us; error 36 us; skew 500 ppm
Aug 09 16:24:00 dist-test-slave-s11t krb5kdc[13156](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292640, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.51.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:24:00.931717 15117 init.cc:377] Logged in from keytab as kudu/127.21.51.130@KRBTEST.COM (short username kudu)
I20260809 16:24:00.932129 15117 webserver.cc:460] Webserver started at http://127.21.51.130:36157/ using document root <none> and password file <none>
I20260809 16:24:00.932358 15117 fs_manager.cc:359] Metadata directory not provided
I20260809 16:24:00.932410 15117 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:24:00.933795 15117 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:24:00.934438 15138 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:24:00.934651 15117 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:24:00.934757 15117 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292417722232-21710-0/minicluster-data/ts-1/data,/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292417722232-21710-0/minicluster-data/ts-1/wal
uuid: "108d298be817495d987fd45a969d96ba"
format_stamp: "Formatted at 2026-08-09 16:23:19 on dist-test-slave-s11t"
server_key: "61136fc1c27ac7e207ba3360818e7bf6"
server_key_iv: "344bc88d0dd8954136aaa8398445bb69"
server_key_version: "encryptionkey@0"
I20260809 16:24:00.935125 15117 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292417722232-21710-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292417722232-21710-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292417722232-21710-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:24:00.947388 15117 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:24:00.949510 15117 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:24:00.949679 15117 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:24:00.949930 15117 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:24:00.950268 15117 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:24:00.950309 15117 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:24:00.950332 15117 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:24:00.950346 15117 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:24:00.957144 15117 rpc_server.cc:307] RPC server started. Bound to: 127.21.51.130:43191
I20260809 16:24:00.957216 15314 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.51.130:43191 every 8 connection(s)
I20260809 16:24:00.957638 15117 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292417722232-21710-0/minicluster-data/ts-1/data/info.pb
I20260809 16:24:00.959842 15321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:00.958825 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:44381 (local address 127.21.51.130:58421)
0809 16:24:00.959547 (+ 722us) negotiation.cc:107] Waiting for socket to connect
0809 16:24:00.959640 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.51.190:44381: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":653,"thread_start_us":88,"threads_started":1}
W20260809 16:24:00.960309 15316 heartbeater.cc:644] Failed to heartbeat to 127.21.51.190:44381 (0 consecutive failures): Network error: Failed to ping master at 127.21.51.190:44381: Client connection negotiation failed: client connection to 127.21.51.190:44381: connect: Connection refused (error 111)
I20260809 16:24:00.960582 15321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:00.960491 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:44381 (local address 127.21.51.130:59831)
0809 16:24:00.960537 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0809 16:24:00.960564 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.51.190:44381: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
I20260809 16:24:00.960852 15321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:00.960796 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:44381 (local address 127.21.51.130:42697)
0809 16:24:00.960820 (+ 24us) negotiation.cc:107] Waiting for socket to connect
0809 16:24:00.960840 (+ 20us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.51.190:44381: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":15}
W20260809 16:24:00.960963 15316 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260809 16:24:00.964895 21710 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrNqUCn/build/release/bin/kudu as pid 15117
I20260809 16:24:00.965205 21710 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrNqUCn/build/release/bin/kudu
/tmp/dist-test-taskrNqUCn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292417722232-21710-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292417722232-21710-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292417722232-21710-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292417722232-21710-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.51.131:40517
--local_ip_for_outbound_sockets=127.21.51.131
--tserver_master_addrs=127.21.51.190:44381
--webserver_port=37545
--webserver_interface=127.21.51.131
--keytab_file=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.131.keytab
--principal=kudu/127.21.51.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.51.148:34103
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292417722232-21710-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292417722232-21710-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292417722232-21710-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:24:01.049157 15322 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:24:01.049369 15322 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:24:01.049393 15322 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:24:01.050563 15322 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:24:01.050609 15322 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:24:01.050637 15322 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.51.131
I20260809 16:24:01.052109 15322 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.51.148:34103
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292417722232-21710-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292417722232-21710-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.131.keytab
--principal=kudu/127.21.51.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.51.131:40517
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292417722232-21710-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.51.131
--webserver_port=37545
--tserver_master_addrs=127.21.51.190:44381
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15322
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.51.131
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292417722232-21710-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:24:01.052359 15322 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:24:01.052597 15322 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:24:01.057664 15334 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:24:01.057745 15337 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:24:01.057735 15333 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:24:01.058027 15322 server_base.cc:1034] running on GCE node
I20260809 16:24:01.058265 15322 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:24:01.058504 15322 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:24:01.059692 15322 hybrid_clock.cc:648] HybridClock initialized: now 1786292641059673 us; error 30 us; skew 500 ppm
Aug 09 16:24:01 dist-test-slave-s11t krb5kdc[13156](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292641, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.51.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:24:01.062479 15322 init.cc:377] Logged in from keytab as kudu/127.21.51.131@KRBTEST.COM (short username kudu)
I20260809 16:24:01.062968 15322 webserver.cc:460] Webserver started at http://127.21.51.131:37545/ using document root <none> and password file <none>
I20260809 16:24:01.063217 15322 fs_manager.cc:359] Metadata directory not provided
I20260809 16:24:01.063285 15322 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:24:01.064656 15322 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:24:01.065312 15346 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:24:01.065492 15322 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 16:24:01.065596 15322 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292417722232-21710-0/minicluster-data/ts-2/data,/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292417722232-21710-0/minicluster-data/ts-2/wal
uuid: "6aa3f90a2d2042e08594f1dd05d7fe30"
format_stamp: "Formatted at 2026-08-09 16:23:19 on dist-test-slave-s11t"
server_key: "3002adbb5d94446f03cb87eaed035f30"
server_key_iv: "e451942046f83051fdc9d91823d2a973"
server_key_version: "encryptionkey@0"
I20260809 16:24:01.065934 15322 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292417722232-21710-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292417722232-21710-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292417722232-21710-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:24:01.073877 15322 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:24:01.075771 15322 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:24:01.075927 15322 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:24:01.076145 15322 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:24:01.076464 15322 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:24:01.076517 15322 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:24:01.076561 15322 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:24:01.076599 15322 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:24:01.082932 15322 rpc_server.cc:307] RPC server started. Bound to: 127.21.51.131:40517
I20260809 16:24:01.082985 15517 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.51.131:40517 every 8 connection(s)
I20260809 16:24:01.083329 15322 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292417722232-21710-0/minicluster-data/ts-2/data/info.pb
I20260809 16:24:01.084316 15523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:01.083951 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:44381 (local address 127.21.51.131:57431)
0809 16:24:01.084106 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0809 16:24:01.084178 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.51.190:44381: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":78,"threads_started":1}
W20260809 16:24:01.084760 15521 heartbeater.cc:644] Failed to heartbeat to 127.21.51.190:44381 (0 consecutive failures): Network error: Failed to ping master at 127.21.51.190:44381: Client connection negotiation failed: client connection to 127.21.51.190:44381: connect: Connection refused (error 111)
I20260809 16:24:01.085042 15523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:01.084968 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:44381 (local address 127.21.51.131:54235)
0809 16:24:01.084998 (+ 30us) negotiation.cc:107] Waiting for socket to connect
0809 16:24:01.085025 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.51.190:44381: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":16}
I20260809 16:24:01.085290 15523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:01.085234 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:44381 (local address 127.21.51.131:54015)
0809 16:24:01.085258 (+ 24us) negotiation.cc:107] Waiting for socket to connect
0809 16:24:01.085278 (+ 20us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.51.190:44381: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":13}
W20260809 16:24:01.085399 15521 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260809 16:24:01.091180 21710 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrNqUCn/build/release/bin/kudu as pid 15322
I20260809 16:24:01.907060 15526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:01.906817 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:44381 (local address 127.21.51.129:44663)
0809 16:24:01.906960 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0809 16:24:01.906996 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.51.190:44381: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":66,"threads_started":1}
I20260809 16:24:01.961575 15527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:01.961287 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:44381 (local address 127.21.51.130:46881)
0809 16:24:01.961449 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0809 16:24:01.961505 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.51.190:44381: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":73,"threads_started":1}
I20260809 16:24:02.085963 15529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:02.085717 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:44381 (local address 127.21.51.131:58045)
0809 16:24:02.085871 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0809 16:24:02.085908 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.51.190:44381: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":102,"threads_started":1}
I20260809 16:24:02.653787 15531 raft_consensus.cc:482] T 8200527efad0406d986bb696bb3394f8 P 7d67bdf64e544c339ca52e32fcd4a5a7 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 16:24:02.653970 15531 raft_consensus.cc:504] T 8200527efad0406d986bb696bb3394f8 P 7d67bdf64e544c339ca52e32fcd4a5a7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "108d298be817495d987fd45a969d96ba" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 43191 } } peers { permanent_uuid: "6aa3f90a2d2042e08594f1dd05d7fe30" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 40517 } } peers { permanent_uuid: "7d67bdf64e544c339ca52e32fcd4a5a7" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 37335 } }
I20260809 16:24:02.654311 15531 leader_election.cc:288] T 8200527efad0406d986bb696bb3394f8 P 7d67bdf64e544c339ca52e32fcd4a5a7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 108d298be817495d987fd45a969d96ba (127.21.51.130:43191), 6aa3f90a2d2042e08594f1dd05d7fe30 (127.21.51.131:40517)
Aug 09 16:24:02 dist-test-slave-s11t krb5kdc[13156](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292640, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.51.129@KRBTEST.COM for kudu/127.21.51.130@KRBTEST.COM
Aug 09 16:24:02 dist-test-slave-s11t krb5kdc[13156](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292640, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.51.129@KRBTEST.COM for kudu/127.21.51.131@KRBTEST.COM
I20260809 16:24:02.664920 15534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:02.654598 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.51.129:46263 (local address 127.21.51.131:40517)
0809 16:24:02.654787 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0809 16:24:02.654797 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 16:24:02.654818 (+ 21us) server_negotiation.cc:407] Connection header received
0809 16:24:02.655258 (+ 440us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:24:02.655260 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:24:02.655302 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:24:02.655460 (+ 158us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:24:02.656321 (+ 861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:24:02.657244 (+ 923us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:24:02.658260 (+ 1016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:24:02.658667 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:24:02.658787 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:24:02.661677 (+ 2890us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:24:02.661680 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:24:02.661680 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:24:02.661689 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:24:02.663527 (+ 1838us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:24:02.663850 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:24:02.663853 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:24:02.663854 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:24:02.663949 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:24:02.664147 (+ 198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:24:02.664148 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:24:02.664149 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:24:02.664372 (+ 223us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:24:02.664456 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0809 16:24:02.664660 (+ 204us) server_negotiation.cc:299] Negotiation successful
0809 16:24:02.664808 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":153,"thread_start_us":66,"threads_started":1}
I20260809 16:24:02.664969 15535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:02.654608 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.51.129:54651 (local address 127.21.51.130:43191)
0809 16:24:02.654787 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0809 16:24:02.654797 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 16:24:02.654817 (+ 20us) server_negotiation.cc:407] Connection header received
0809 16:24:02.655258 (+ 441us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:24:02.655260 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:24:02.655302 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:24:02.655454 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:24:02.656286 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:24:02.657243 (+ 957us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:24:02.658266 (+ 1023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:24:02.658666 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:24:02.658787 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:24:02.661226 (+ 2439us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:24:02.661230 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:24:02.661230 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:24:02.661240 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:24:02.663279 (+ 2039us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:24:02.663691 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:24:02.663694 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:24:02.663695 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:24:02.663789 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:24:02.664037 (+ 248us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:24:02.664038 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:24:02.664038 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:24:02.664276 (+ 238us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:24:02.664362 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0809 16:24:02.664653 (+ 291us) server_negotiation.cc:299] Negotiation successful
0809 16:24:02.664830 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":132,"thread_start_us":60,"threads_started":1}
I20260809 16:24:02.664919 15532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:02.654465 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.131:40517 (local address 127.21.51.129:46263)
0809 16:24:02.654571 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0809 16:24:02.654582 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 16:24:02.654644 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:24:02.655508 (+ 864us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:24:02.655512 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:24:02.655972 (+ 460us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:24:02.656168 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:24:02.656174 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:24:02.657346 (+ 1172us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:24:02.657347 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:24:02.658165 (+ 818us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:24:02.658167 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:24:02.658759 (+ 592us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:24:02.658760 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:24:02.658877 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:24:02.659447 (+ 570us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:24:02.659455 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:24:02.661570 (+ 2115us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:24:02.663604 (+ 2034us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:24:02.663605 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:24:02.663606 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:24:02.663773 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:24:02.664023 (+ 250us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:24:02.664024 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:24:02.664025 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:24:02.664087 (+ 62us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:24:02.664451 (+ 364us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:24:02.664452 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:24:02.664497 (+ 45us) client_negotiation.cc:769] Sending connection context
0809 16:24:02.664584 (+ 87us) client_negotiation.cc:240] Negotiation successful
0809 16:24:02.664781 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":92,"mutex_wait_us":54,"thread_start_us":82,"threads_started":1}
I20260809 16:24:02.665719 15533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:02.654465 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.130:43191 (local address 127.21.51.129:54651)
0809 16:24:02.654615 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0809 16:24:02.654624 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 16:24:02.654665 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:24:02.655510 (+ 845us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:24:02.655513 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:24:02.655974 (+ 461us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:24:02.656168 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:24:02.656173 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:24:02.657343 (+ 1170us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:24:02.657344 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:24:02.658165 (+ 821us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:24:02.658168 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:24:02.658762 (+ 594us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:24:02.658763 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:24:02.658877 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:24:02.659512 (+ 635us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:24:02.659516 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:24:02.661123 (+ 1607us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:24:02.663359 (+ 2236us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:24:02.663365 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:24:02.663366 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:24:02.663582 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:24:02.663858 (+ 276us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:24:02.663859 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:24:02.663859 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:24:02.663967 (+ 108us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:24:02.664354 (+ 387us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:24:02.664361 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:24:02.664486 (+ 125us) client_negotiation.cc:769] Sending connection context
0809 16:24:02.664584 (+ 98us) client_negotiation.cc:240] Negotiation successful
0809 16:24:02.664860 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"mutex_wait_us":167,"thread_start_us":48,"threads_started":1}
W20260809 16:24:02.665907 14940 leader_election.cc:341] T 8200527efad0406d986bb696bb3394f8 P 7d67bdf64e544c339ca52e32fcd4a5a7 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 6aa3f90a2d2042e08594f1dd05d7fe30 (127.21.51.131:40517): Not found: Tablet not found: 8200527efad0406d986bb696bb3394f8
W20260809 16:24:02.666369 14939 leader_election.cc:341] T 8200527efad0406d986bb696bb3394f8 P 7d67bdf64e544c339ca52e32fcd4a5a7 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 108d298be817495d987fd45a969d96ba (127.21.51.130:43191): Not found: Tablet not found: 8200527efad0406d986bb696bb3394f8
I20260809 16:24:02.666435 14939 leader_election.cc:302] T 8200527efad0406d986bb696bb3394f8 P 7d67bdf64e544c339ca52e32fcd4a5a7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d67bdf64e544c339ca52e32fcd4a5a7; no voters: 108d298be817495d987fd45a969d96ba, 6aa3f90a2d2042e08594f1dd05d7fe30
I20260809 16:24:02.666568 15531 raft_consensus.cc:2726] T 8200527efad0406d986bb696bb3394f8 P 7d67bdf64e544c339ca52e32fcd4a5a7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 16:24:02.907682 15533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:02.907512 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:44381 (local address 127.21.51.129:47881)
0809 16:24:02.907595 (+ 83us) negotiation.cc:107] Waiting for socket to connect
0809 16:24:02.907639 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.51.190:44381: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260809 16:24:02.962304 15536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:02.962034 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:44381 (local address 127.21.51.130:60071)
0809 16:24:02.962198 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0809 16:24:02.962256 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.51.190:44381: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":85,"threads_started":1}
I20260809 16:24:03.086658 15537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:03.086410 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:44381 (local address 127.21.51.131:49359)
0809 16:24:03.086563 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0809 16:24:03.086614 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.51.190:44381: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":80,"threads_started":1}
I20260809 16:24:03.908427 15538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:03.908178 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:44381 (local address 127.21.51.129:54653)
0809 16:24:03.908340 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0809 16:24:03.908379 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.51.190:44381: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":91,"threads_started":1}
I20260809 16:24:03.963039 15539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:03.962786 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:44381 (local address 127.21.51.130:59235)
0809 16:24:03.962956 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0809 16:24:03.962995 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.51.190:44381: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":85,"threads_started":1}
I20260809 16:24:04.087430 15541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:04.087210 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:44381 (local address 127.21.51.131:46967)
0809 16:24:04.087345 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0809 16:24:04.087386 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.51.190:44381: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":66,"threads_started":1}
I20260809 16:24:04.285791 15542 raft_consensus.cc:482] T 8200527efad0406d986bb696bb3394f8 P 7d67bdf64e544c339ca52e32fcd4a5a7 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 16:24:04.285918 15542 raft_consensus.cc:504] T 8200527efad0406d986bb696bb3394f8 P 7d67bdf64e544c339ca52e32fcd4a5a7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "108d298be817495d987fd45a969d96ba" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 43191 } } peers { permanent_uuid: "6aa3f90a2d2042e08594f1dd05d7fe30" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 40517 } } peers { permanent_uuid: "7d67bdf64e544c339ca52e32fcd4a5a7" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 37335 } }
I20260809 16:24:04.286123 15542 leader_election.cc:288] T 8200527efad0406d986bb696bb3394f8 P 7d67bdf64e544c339ca52e32fcd4a5a7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 108d298be817495d987fd45a969d96ba (127.21.51.130:43191), 6aa3f90a2d2042e08594f1dd05d7fe30 (127.21.51.131:40517)
W20260809 16:24:04.286562 14939 leader_election.cc:341] T 8200527efad0406d986bb696bb3394f8 P 7d67bdf64e544c339ca52e32fcd4a5a7 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 108d298be817495d987fd45a969d96ba (127.21.51.130:43191): Not found: Tablet not found: 8200527efad0406d986bb696bb3394f8
W20260809 16:24:04.286644 14940 leader_election.cc:341] T 8200527efad0406d986bb696bb3394f8 P 7d67bdf64e544c339ca52e32fcd4a5a7 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 6aa3f90a2d2042e08594f1dd05d7fe30 (127.21.51.131:40517): Not found: Tablet not found: 8200527efad0406d986bb696bb3394f8
I20260809 16:24:04.286717 14940 leader_election.cc:302] T 8200527efad0406d986bb696bb3394f8 P 7d67bdf64e544c339ca52e32fcd4a5a7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d67bdf64e544c339ca52e32fcd4a5a7; no voters: 108d298be817495d987fd45a969d96ba, 6aa3f90a2d2042e08594f1dd05d7fe30
I20260809 16:24:04.286777 15542 raft_consensus.cc:2726] T 8200527efad0406d986bb696bb3394f8 P 7d67bdf64e544c339ca52e32fcd4a5a7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 16:24:04.909176 15544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:04.908962 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:44381 (local address 127.21.51.129:36111)
0809 16:24:04.909088 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0809 16:24:04.909126 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.51.190:44381: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":64,"threads_started":1}
I20260809 16:24:04.963735 15545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:04.963529 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:44381 (local address 127.21.51.130:45717)
0809 16:24:04.963653 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0809 16:24:04.963691 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.51.190:44381: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":58,"threads_started":1}
I20260809 16:24:05.088121 15547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:05.087911 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:44381 (local address 127.21.51.131:46199)
0809 16:24:05.088029 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0809 16:24:05.088063 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.51.190:44381: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":65,"threads_started":1}
I20260809 16:24:05.909915 15549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:05.909691 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:44381 (local address 127.21.51.129:53775)
0809 16:24:05.909825 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0809 16:24:05.909863 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.51.190:44381: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":63,"threads_started":1}
I20260809 16:24:05.964466 15550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:05.964224 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:44381 (local address 127.21.51.130:33591)
0809 16:24:05.964370 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0809 16:24:05.964424 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.51.190:44381: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":64,"threads_started":1}
I20260809 16:24:06.088855 15552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:06.088595 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:44381 (local address 127.21.51.131:37031)
0809 16:24:06.088771 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0809 16:24:06.088811 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.51.190:44381: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":59,"threads_started":1}
I20260809 16:24:06.355535 15553 raft_consensus.cc:482] T 8200527efad0406d986bb696bb3394f8 P 7d67bdf64e544c339ca52e32fcd4a5a7 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 16:24:06.355634 15553 raft_consensus.cc:504] T 8200527efad0406d986bb696bb3394f8 P 7d67bdf64e544c339ca52e32fcd4a5a7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "108d298be817495d987fd45a969d96ba" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 43191 } } peers { permanent_uuid: "6aa3f90a2d2042e08594f1dd05d7fe30" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 40517 } } peers { permanent_uuid: "7d67bdf64e544c339ca52e32fcd4a5a7" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 37335 } }
I20260809 16:24:06.355849 15553 leader_election.cc:288] T 8200527efad0406d986bb696bb3394f8 P 7d67bdf64e544c339ca52e32fcd4a5a7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 108d298be817495d987fd45a969d96ba (127.21.51.130:43191), 6aa3f90a2d2042e08594f1dd05d7fe30 (127.21.51.131:40517)
W20260809 16:24:06.356238 14939 leader_election.cc:341] T 8200527efad0406d986bb696bb3394f8 P 7d67bdf64e544c339ca52e32fcd4a5a7 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 108d298be817495d987fd45a969d96ba (127.21.51.130:43191): Not found: Tablet not found: 8200527efad0406d986bb696bb3394f8
W20260809 16:24:06.356346 14940 leader_election.cc:341] T 8200527efad0406d986bb696bb3394f8 P 7d67bdf64e544c339ca52e32fcd4a5a7 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 6aa3f90a2d2042e08594f1dd05d7fe30 (127.21.51.131:40517): Not found: Tablet not found: 8200527efad0406d986bb696bb3394f8
I20260809 16:24:06.356395 14940 leader_election.cc:302] T 8200527efad0406d986bb696bb3394f8 P 7d67bdf64e544c339ca52e32fcd4a5a7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d67bdf64e544c339ca52e32fcd4a5a7; no voters: 108d298be817495d987fd45a969d96ba, 6aa3f90a2d2042e08594f1dd05d7fe30
I20260809 16:24:06.356482 15553 raft_consensus.cc:2726] T 8200527efad0406d986bb696bb3394f8 P 7d67bdf64e544c339ca52e32fcd4a5a7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 16:24:06.910660 15555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:06.910427 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:44381 (local address 127.21.51.129:55925)
0809 16:24:06.910578 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0809 16:24:06.910613 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.51.190:44381: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":88,"threads_started":1}
I20260809 16:24:06.965128 15556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:06.964904 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:44381 (local address 127.21.51.130:56095)
0809 16:24:06.965048 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0809 16:24:06.965086 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.51.190:44381: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":82,"threads_started":1}
I20260809 16:24:07.089586 15557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:07.089345 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:44381 (local address 127.21.51.131:54987)
0809 16:24:07.089505 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0809 16:24:07.089543 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.51.190:44381: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":67,"threads_started":1}
I20260809 16:24:07.911464 15558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:07.911234 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:44381 (local address 127.21.51.129:57675)
0809 16:24:07.911368 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0809 16:24:07.911416 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.51.190:44381: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":73,"threads_started":1}
I20260809 16:24:07.965879 15559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:07.965608 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:44381 (local address 127.21.51.130:49791)
0809 16:24:07.965794 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0809 16:24:07.965835 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.51.190:44381: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":69,"threads_started":1}
I20260809 16:24:08.090399 15560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:08.090139 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:44381 (local address 127.21.51.131:49091)
0809 16:24:08.090308 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0809 16:24:08.090353 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.51.190:44381: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":70,"threads_started":1}
I20260809 16:24:08.912199 15562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:08.911956 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:44381 (local address 127.21.51.129:39759)
0809 16:24:08.912112 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0809 16:24:08.912151 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.51.190:44381: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":70,"threads_started":1}
I20260809 16:24:08.966600 15563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:08.966384 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:44381 (local address 127.21.51.130:52543)
0809 16:24:08.966515 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0809 16:24:08.966555 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.51.190:44381: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":68,"threads_started":1}
I20260809 16:24:09.091250 15565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:09.090971 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:44381 (local address 127.21.51.131:54547)
0809 16:24:09.091153 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0809 16:24:09.091196 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.51.190:44381: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":87,"threads_started":1}
I20260809 16:24:09.913029 15566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:09.912767 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:44381 (local address 127.21.51.129:45807)
0809 16:24:09.912902 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0809 16:24:09.912982 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.51.190:44381: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":64,"threads_started":1}
I20260809 16:24:09.967248 15567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:09.967031 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:44381 (local address 127.21.51.130:33631)
0809 16:24:09.967165 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0809 16:24:09.967201 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.51.190:44381: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":60,"threads_started":1}
I20260809 16:24:10.091954 15568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:10.091710 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:44381 (local address 127.21.51.131:53745)
0809 16:24:10.091873 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0809 16:24:10.091905 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.51.190:44381: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":94,"threads_started":1}
I20260809 16:24:10.389776 15569 raft_consensus.cc:482] T 8200527efad0406d986bb696bb3394f8 P 7d67bdf64e544c339ca52e32fcd4a5a7 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 16:24:10.389871 15569 raft_consensus.cc:504] T 8200527efad0406d986bb696bb3394f8 P 7d67bdf64e544c339ca52e32fcd4a5a7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "108d298be817495d987fd45a969d96ba" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 43191 } } peers { permanent_uuid: "6aa3f90a2d2042e08594f1dd05d7fe30" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 40517 } } peers { permanent_uuid: "7d67bdf64e544c339ca52e32fcd4a5a7" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 37335 } }
I20260809 16:24:10.390096 15569 leader_election.cc:288] T 8200527efad0406d986bb696bb3394f8 P 7d67bdf64e544c339ca52e32fcd4a5a7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 108d298be817495d987fd45a969d96ba (127.21.51.130:43191), 6aa3f90a2d2042e08594f1dd05d7fe30 (127.21.51.131:40517)
W20260809 16:24:10.390384 14940 leader_election.cc:341] T 8200527efad0406d986bb696bb3394f8 P 7d67bdf64e544c339ca52e32fcd4a5a7 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 6aa3f90a2d2042e08594f1dd05d7fe30 (127.21.51.131:40517): Not found: Tablet not found: 8200527efad0406d986bb696bb3394f8
W20260809 16:24:10.390472 14939 leader_election.cc:341] T 8200527efad0406d986bb696bb3394f8 P 7d67bdf64e544c339ca52e32fcd4a5a7 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 108d298be817495d987fd45a969d96ba (127.21.51.130:43191): Not found: Tablet not found: 8200527efad0406d986bb696bb3394f8
I20260809 16:24:10.390547 14939 leader_election.cc:302] T 8200527efad0406d986bb696bb3394f8 P 7d67bdf64e544c339ca52e32fcd4a5a7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d67bdf64e544c339ca52e32fcd4a5a7; no voters: 108d298be817495d987fd45a969d96ba, 6aa3f90a2d2042e08594f1dd05d7fe30
I20260809 16:24:10.390633 15569 raft_consensus.cc:2726] T 8200527efad0406d986bb696bb3394f8 P 7d67bdf64e544c339ca52e32fcd4a5a7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 16:24:10.913727 15570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:10.913519 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:44381 (local address 127.21.51.129:56023)
0809 16:24:10.913645 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0809 16:24:10.913682 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.51.190:44381: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":68,"threads_started":1}
I20260809 16:24:10.967965 15571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:10.967710 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:44381 (local address 127.21.51.130:55989)
0809 16:24:10.967875 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0809 16:24:10.967917 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.51.190:44381: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":67,"threads_started":1}
I20260809 16:24:11.092653 15572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:11.092397 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:44381 (local address 127.21.51.131:40629)
0809 16:24:11.092563 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0809 16:24:11.092605 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.51.190:44381: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":85,"threads_started":1}
I20260809 16:24:11.914433 15574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:11.914194 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:44381 (local address 127.21.51.129:49205)
0809 16:24:11.914349 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0809 16:24:11.914387 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.51.190:44381: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":69,"threads_started":1}
I20260809 16:24:11.968791 15575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:11.968520 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:44381 (local address 127.21.51.130:59489)
0809 16:24:11.968694 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0809 16:24:11.968743 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.51.190:44381: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":69,"threads_started":1}
I20260809 16:24:12.093389 15576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:12.093168 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:44381 (local address 127.21.51.131:37899)
0809 16:24:12.093294 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0809 16:24:12.093332 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.51.190:44381: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":66,"threads_started":1}
I20260809 16:24:12.915215 15578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:12.914952 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:44381 (local address 127.21.51.129:45293)
0809 16:24:12.915114 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0809 16:24:12.915168 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.51.190:44381: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":66,"threads_started":1}
I20260809 16:24:12.969611 15580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:12.969336 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:44381 (local address 127.21.51.130:56211)
0809 16:24:12.969500 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0809 16:24:12.969562 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.51.190:44381: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":69,"threads_started":1}
I20260809 16:24:13.094307 15581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:13.094031 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:44381 (local address 127.21.51.131:42615)
0809 16:24:13.094209 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0809 16:24:13.094260 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.51.190:44381: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":85,"threads_started":1}
I20260809 16:24:13.916122 15582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:13.915866 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:44381 (local address 127.21.51.129:50273)
0809 16:24:13.916017 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0809 16:24:13.916068 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.51.190:44381: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":68,"threads_started":1}
I20260809 16:24:13.970425 15583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:13.970169 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:44381 (local address 127.21.51.130:56897)
0809 16:24:13.970328 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0809 16:24:13.970379 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.51.190:44381: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":66,"threads_started":1}
I20260809 16:24:14.095145 15586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:14.094882 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:44381 (local address 127.21.51.131:39617)
0809 16:24:14.095040 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0809 16:24:14.095097 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.51.190:44381: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":86,"threads_started":1}
I20260809 16:24:14.708498 15587 raft_consensus.cc:482] T 8200527efad0406d986bb696bb3394f8 P 7d67bdf64e544c339ca52e32fcd4a5a7 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 16:24:14.708595 15587 raft_consensus.cc:504] T 8200527efad0406d986bb696bb3394f8 P 7d67bdf64e544c339ca52e32fcd4a5a7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "108d298be817495d987fd45a969d96ba" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 43191 } } peers { permanent_uuid: "6aa3f90a2d2042e08594f1dd05d7fe30" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 40517 } } peers { permanent_uuid: "7d67bdf64e544c339ca52e32fcd4a5a7" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 37335 } }
I20260809 16:24:14.708837 15587 leader_election.cc:288] T 8200527efad0406d986bb696bb3394f8 P 7d67bdf64e544c339ca52e32fcd4a5a7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 108d298be817495d987fd45a969d96ba (127.21.51.130:43191), 6aa3f90a2d2042e08594f1dd05d7fe30 (127.21.51.131:40517)
W20260809 16:24:14.709271 14939 leader_election.cc:341] T 8200527efad0406d986bb696bb3394f8 P 7d67bdf64e544c339ca52e32fcd4a5a7 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 108d298be817495d987fd45a969d96ba (127.21.51.130:43191): Not found: Tablet not found: 8200527efad0406d986bb696bb3394f8
W20260809 16:24:14.709360 14940 leader_election.cc:341] T 8200527efad0406d986bb696bb3394f8 P 7d67bdf64e544c339ca52e32fcd4a5a7 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 6aa3f90a2d2042e08594f1dd05d7fe30 (127.21.51.131:40517): Not found: Tablet not found: 8200527efad0406d986bb696bb3394f8
I20260809 16:24:14.709458 14940 leader_election.cc:302] T 8200527efad0406d986bb696bb3394f8 P 7d67bdf64e544c339ca52e32fcd4a5a7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d67bdf64e544c339ca52e32fcd4a5a7; no voters: 108d298be817495d987fd45a969d96ba, 6aa3f90a2d2042e08594f1dd05d7fe30
I20260809 16:24:14.709555 15587 raft_consensus.cc:2726] T 8200527efad0406d986bb696bb3394f8 P 7d67bdf64e544c339ca52e32fcd4a5a7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 16:24:14.916966 15588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:14.916651 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:44381 (local address 127.21.51.129:39615)
0809 16:24:14.916878 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0809 16:24:14.916920 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.51.190:44381: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":208,"thread_start_us":130,"threads_started":1}
I20260809 16:24:14.971230 15589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:14.971015 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:44381 (local address 127.21.51.130:45313)
0809 16:24:14.971149 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0809 16:24:14.971185 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.51.190:44381: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":64,"threads_started":1}
I20260809 16:24:15.095839 15590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:15.095577 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:44381 (local address 127.21.51.131:34565)
0809 16:24:15.095753 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0809 16:24:15.095792 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.51.190:44381: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":95,"threads_started":1}
I20260809 16:24:15.917802 15591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:15.917573 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:44381 (local address 127.21.51.129:46627)
0809 16:24:15.917712 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0809 16:24:15.917753 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.51.190:44381: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":62,"threads_started":1}
I20260809 16:24:15.972071 15592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:15.971812 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:44381 (local address 127.21.51.130:57837)
0809 16:24:15.971968 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0809 16:24:15.972026 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.51.190:44381: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":64,"threads_started":1}
I20260809 16:24:16.096588 15593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:16.096374 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:44381 (local address 127.21.51.131:36065)
0809 16:24:16.096503 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0809 16:24:16.096541 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.51.190:44381: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":65,"threads_started":1}
I20260809 16:24:16.097708 21710 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrNqUCn/build/release/bin/kudu with pid 14867
I20260809 16:24:16.109565 21710 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrNqUCn/build/release/bin/kudu with pid 15117
I20260809 16:24:16.118853 21710 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrNqUCn/build/release/bin/kudu with pid 15322
2026-08-09T16:24:16Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (58636 ms)
[----------] 30 tests from SecurityITest (238405 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786292417722232-21710-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 16:24:16 dist-test-slave-s11t krb5kdc[15596](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 16:24:16 dist-test-slave-s11t krb5kdc[15596](info): set up 2 sockets
Aug 09 16:24:16 dist-test-slave-s11t krb5kdc[15596](info): commencing operation
krb5kdc: starting...
W20260809 16:24:17.392024 21710 mini_kdc.cc:121] Time spent starting KDC: real 1.250s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786292417722232-21710-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786292417722232-21710-0/krb5kdc/test-user.keytab.
Aug 09 16:24:17 dist-test-slave-s11t krb5kdc[15596](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292657, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T16:24:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T16:24:17Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.51.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.51.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.51.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.190.keytab.
Entry for principal kudu/127.21.51.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.190.keytab.
I20260809 16:24:17.514946 21710 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrNqUCn/build/release/bin/kudu
/tmp/dist-test-taskrNqUCn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786292417722232-21710-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786292417722232-21710-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786292417722232-21710-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786292417722232-21710-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.51.190:39581
--webserver_interface=127.21.51.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.190.keytab
--principal=kudu/127.21.51.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.51.148:34019
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.51.190:39581
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786292417722232-21710-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786292417722232-21710-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786292417722232-21710-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:24:17.596642 15612 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:24:17.596868 15612 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:24:17.596894 15612 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:24:17.597983 15612 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 16:24:17.598021 15612 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:24:17.598035 15612 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:24:17.598048 15612 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 16:24:17.598060 15612 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 16:24:17.599466 15612 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.51.148:34019
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786292417722232-21710-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786292417722232-21710-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.51.190:39581
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.190.keytab
--principal=kudu/127.21.51.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.51.190:39581
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786292417722232-21710-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.51.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15612
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786292417722232-21710-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:24:17.599663 15612 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:24:17.599828 15612 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:24:17.604871 15621 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:24:17.604930 15612 server_base.cc:1034] running on GCE node
W20260809 16:24:17.604930 15618 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:24:17.604888 15619 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:24:17.605504 15612 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:24:17.605779 15612 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:24:17.606927 15612 hybrid_clock.cc:648] HybridClock initialized: now 1786292657606917 us; error 30 us; skew 500 ppm
Aug 09 16:24:17 dist-test-slave-s11t krb5kdc[15596](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292657, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.51.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:24:17.609714 15612 init.cc:377] Logged in from keytab as kudu/127.21.51.190@KRBTEST.COM (short username kudu)
I20260809 16:24:17.610244 15612 webserver.cc:460] Webserver started at http://127.21.51.190:36765/ using document root <none> and password file <none>
I20260809 16:24:17.610503 15612 fs_manager.cc:359] Metadata directory not provided
I20260809 16:24:17.610575 15612 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:24:17.610728 15612 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:24:17.611796 15612 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786292417722232-21710-0/minicluster-data/master-0/data/instance:
uuid: "31245ca59a7c4c0485e7f0fe141a06ca"
format_stamp: "Formatted at 2026-08-09 16:24:17 on dist-test-slave-s11t"
server_key: "a672f68c7c5e325a1180bd23bff13573"
server_key_iv: "406c92042104cc8618df18c9b3e3c0a9"
server_key_version: "encryptionkey@0"
I20260809 16:24:17.612159 15612 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786292417722232-21710-0/minicluster-data/master-0/wal/instance:
uuid: "31245ca59a7c4c0485e7f0fe141a06ca"
format_stamp: "Formatted at 2026-08-09 16:24:17 on dist-test-slave-s11t"
server_key: "a672f68c7c5e325a1180bd23bff13573"
server_key_iv: "406c92042104cc8618df18c9b3e3c0a9"
server_key_version: "encryptionkey@0"
I20260809 16:24:17.613682 15612 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 16:24:17.614490 15634 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:24:17.614714 15612 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 16:24:17.614822 15612 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786292417722232-21710-0/minicluster-data/master-0/data,/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786292417722232-21710-0/minicluster-data/master-0/wal
uuid: "31245ca59a7c4c0485e7f0fe141a06ca"
format_stamp: "Formatted at 2026-08-09 16:24:17 on dist-test-slave-s11t"
server_key: "a672f68c7c5e325a1180bd23bff13573"
server_key_iv: "406c92042104cc8618df18c9b3e3c0a9"
server_key_version: "encryptionkey@0"
I20260809 16:24:17.614913 15612 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786292417722232-21710-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786292417722232-21710-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786292417722232-21710-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:24:17.637653 15612 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:24:17.639604 15612 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:24:17.639780 15612 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:24:17.644361 15612 rpc_server.cc:307] RPC server started. Bound to: 127.21.51.190:39581
I20260809 16:24:17.644402 15721 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.51.190:39581 every 8 connection(s)
I20260809 16:24:17.644809 15612 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786292417722232-21710-0/minicluster-data/master-0/data/info.pb
I20260809 16:24:17.645702 15722 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:24:17.648001 15722 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 31245ca59a7c4c0485e7f0fe141a06ca: Bootstrap starting.
I20260809 16:24:17.648588 15722 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 31245ca59a7c4c0485e7f0fe141a06ca: Neither blocks nor log segments found. Creating new log.
I20260809 16:24:17.648918 15722 log.cc:824] T 00000000000000000000000000000000 P 31245ca59a7c4c0485e7f0fe141a06ca: Log is configured to *not* fsync() on all Append() calls
I20260809 16:24:17.649647 15722 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 31245ca59a7c4c0485e7f0fe141a06ca: No bootstrap required, opened a new log
I20260809 16:24:17.649950 21710 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrNqUCn/build/release/bin/kudu as pid 15612
I20260809 16:24:17.650094 21710 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786292417722232-21710-0/minicluster-data/master-0/wal/instance
I20260809 16:24:17.650303 21710 external_mini_cluster.cc:1428] Setting key 8c58dca6567418703baa970995db1f59
I20260809 16:24:17.651584 15722 raft_consensus.cc:348] T 00000000000000000000000000000000 P 31245ca59a7c4c0485e7f0fe141a06ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "31245ca59a7c4c0485e7f0fe141a06ca" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 39581 } }
I20260809 16:24:17.651743 15722 raft_consensus.cc:374] T 00000000000000000000000000000000 P 31245ca59a7c4c0485e7f0fe141a06ca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:24:17.651832 15722 raft_consensus.cc:729] T 00000000000000000000000000000000 P 31245ca59a7c4c0485e7f0fe141a06ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 31245ca59a7c4c0485e7f0fe141a06ca, State: Initialized, Role: FOLLOWER
I20260809 16:24:17.652019 15722 consensus_queue.cc:260] T 00000000000000000000000000000000 P 31245ca59a7c4c0485e7f0fe141a06ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "31245ca59a7c4c0485e7f0fe141a06ca" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 39581 } }
I20260809 16:24:17.652118 15722 raft_consensus.cc:388] T 00000000000000000000000000000000 P 31245ca59a7c4c0485e7f0fe141a06ca [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 16:24:17.652164 15722 raft_consensus.cc:482] T 00000000000000000000000000000000 P 31245ca59a7c4c0485e7f0fe141a06ca [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 16:24:17.652212 15722 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 31245ca59a7c4c0485e7f0fe141a06ca [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:24:17.653133 15722 raft_consensus.cc:504] T 00000000000000000000000000000000 P 31245ca59a7c4c0485e7f0fe141a06ca [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "31245ca59a7c4c0485e7f0fe141a06ca" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 39581 } }
I20260809 16:24:17.653254 15722 leader_election.cc:302] T 00000000000000000000000000000000 P 31245ca59a7c4c0485e7f0fe141a06ca [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 31245ca59a7c4c0485e7f0fe141a06ca; no voters:
I20260809 16:24:17.653435 15722 leader_election.cc:288] T 00000000000000000000000000000000 P 31245ca59a7c4c0485e7f0fe141a06ca [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 16:24:17.653486 15729 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 31245ca59a7c4c0485e7f0fe141a06ca [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:24:17.653637 15729 raft_consensus.cc:686] T 00000000000000000000000000000000 P 31245ca59a7c4c0485e7f0fe141a06ca [term 1 LEADER]: Becoming Leader. State: Replica: 31245ca59a7c4c0485e7f0fe141a06ca, State: Running, Role: LEADER
I20260809 16:24:17.653697 15722 sys_catalog.cc:564] T 00000000000000000000000000000000 P 31245ca59a7c4c0485e7f0fe141a06ca [sys.catalog]: configured and running, proceeding with master startup.
I20260809 16:24:17.653776 15729 consensus_queue.cc:237] T 00000000000000000000000000000000 P 31245ca59a7c4c0485e7f0fe141a06ca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "31245ca59a7c4c0485e7f0fe141a06ca" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 39581 } }
I20260809 16:24:17.655030 15730 sys_catalog.cc:455] T 00000000000000000000000000000000 P 31245ca59a7c4c0485e7f0fe141a06ca [sys.catalog]: SysCatalogTable state changed. Reason: New leader 31245ca59a7c4c0485e7f0fe141a06ca. Latest consensus state: current_term: 1 leader_uuid: "31245ca59a7c4c0485e7f0fe141a06ca" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "31245ca59a7c4c0485e7f0fe141a06ca" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 39581 } } }
I20260809 16:24:17.655115 15730 sys_catalog.cc:458] T 00000000000000000000000000000000 P 31245ca59a7c4c0485e7f0fe141a06ca [sys.catalog]: This master's current role is: LEADER
I20260809 16:24:17.655398 15734 sys_catalog.cc:455] T 00000000000000000000000000000000 P 31245ca59a7c4c0485e7f0fe141a06ca [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "31245ca59a7c4c0485e7f0fe141a06ca" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "31245ca59a7c4c0485e7f0fe141a06ca" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 39581 } } }
W20260809 16:24:17.655613 15755 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 31245ca59a7c4c0485e7f0fe141a06ca: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
I20260809 16:24:17.655678 15734 sys_catalog.cc:458] T 00000000000000000000000000000000 P 31245ca59a7c4c0485e7f0fe141a06ca [sys.catalog]: This master's current role is: LEADER
W20260809 16:24:17.655798 15755 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 16:24:17.655884 15756 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 16:24:17.656040 15756 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 16:24:17.657598 15756 catalog_manager.cc:1347] Generated new cluster ID: 5e9040c948c14479a432934beb250c4c
I20260809 16:24:17.657657 15756 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 09 16:24:17 dist-test-slave-s11t krb5kdc[15596](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292657, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.51.190@KRBTEST.COM
I20260809 16:24:17.661820 15727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:17.650837 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45616 (local address 127.21.51.190:39581)
0809 16:24:17.651022 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0809 16:24:17.651032 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 16:24:17.651064 (+ 32us) server_negotiation.cc:407] Connection header received
0809 16:24:17.651569 (+ 505us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:24:17.651575 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:24:17.651622 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:24:17.651760 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:24:17.652558 (+ 798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:24:17.653760 (+ 1202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:24:17.654918 (+ 1158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:24:17.655577 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:24:17.655685 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:24:17.658523 (+ 2838us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:24:17.658527 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:24:17.658528 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:24:17.658539 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:24:17.660199 (+ 1660us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:24:17.660567 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:24:17.660569 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:24:17.660570 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:24:17.660734 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:24:17.660974 (+ 240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:24:17.660975 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:24:17.660975 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:24:17.661238 (+ 263us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:24:17.661443 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0809 16:24:17.661510 (+ 67us) server_negotiation.cc:299] Negotiation successful
0809 16:24:17.661655 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":131,"thread_start_us":83,"threads_started":1}
I20260809 16:24:17.675020 15756 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 16:24:17.675557 15756 catalog_manager.cc:1504] Loading token signing keys...
I20260809 16:24:17.682752 15756 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 31245ca59a7c4c0485e7f0fe141a06ca: Generated new TSK 0
I20260809 16:24:17.682950 15756 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.21.51.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.51.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.51.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.129.keytab.
Entry for principal kudu/127.21.51.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.129.keytab.
I20260809 16:24:17.715047 21710 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrNqUCn/build/release/bin/kudu
/tmp/dist-test-taskrNqUCn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786292417722232-21710-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786292417722232-21710-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786292417722232-21710-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786292417722232-21710-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.51.129:0
--local_ip_for_outbound_sockets=127.21.51.129
--webserver_interface=127.21.51.129
--webserver_port=0
--tserver_master_addrs=127.21.51.190:39581
--keytab_file=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.129.keytab
--principal=kudu/127.21.51.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.51.148:34019
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786292417722232-21710-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786292417722232-21710-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786292417722232-21710-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:24:17.806143 15763 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:24:17.806365 15763 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:24:17.806392 15763 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:24:17.807611 15763 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:24:17.807655 15763 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:24:17.807685 15763 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.51.129
I20260809 16:24:17.809342 15763 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.51.148:34019
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786292417722232-21710-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786292417722232-21710-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.129.keytab
--principal=kudu/127.21.51.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.51.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786292417722232-21710-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.51.129
--webserver_port=0
--tserver_master_addrs=127.21.51.190:39581
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15763
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.51.129
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786292417722232-21710-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:24:17.809602 15763 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:24:17.809844 15763 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:24:17.815466 15771 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:24:17.815514 15776 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:24:17.815485 15770 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:24:17.815732 15763 server_base.cc:1034] running on GCE node
I20260809 16:24:17.815948 15763 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:24:17.816243 15763 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:24:17.817406 15763 hybrid_clock.cc:648] HybridClock initialized: now 1786292657817387 us; error 41 us; skew 500 ppm
Aug 09 16:24:17 dist-test-slave-s11t krb5kdc[15596](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292657, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.51.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:24:17.820120 15763 init.cc:377] Logged in from keytab as kudu/127.21.51.129@KRBTEST.COM (short username kudu)
I20260809 16:24:17.820636 15763 webserver.cc:460] Webserver started at http://127.21.51.129:42395/ using document root <none> and password file <none>
I20260809 16:24:17.820940 15763 fs_manager.cc:359] Metadata directory not provided
I20260809 16:24:17.820993 15763 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:24:17.821171 15763 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:24:17.822242 15763 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786292417722232-21710-0/minicluster-data/ts-0/data/instance:
uuid: "0c28fcd823f4440aa2a17070dcdb3f19"
format_stamp: "Formatted at 2026-08-09 16:24:17 on dist-test-slave-s11t"
server_key: "3e45f0b02719e0215e4bd28e27db09e1"
server_key_iv: "3743e7069633f504a27f083f3c0ccc69"
server_key_version: "encryptionkey@0"
I20260809 16:24:17.822731 15763 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786292417722232-21710-0/minicluster-data/ts-0/wal/instance:
uuid: "0c28fcd823f4440aa2a17070dcdb3f19"
format_stamp: "Formatted at 2026-08-09 16:24:17 on dist-test-slave-s11t"
server_key: "3e45f0b02719e0215e4bd28e27db09e1"
server_key_iv: "3743e7069633f504a27f083f3c0ccc69"
server_key_version: "encryptionkey@0"
I20260809 16:24:17.824141 15763 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 16:24:17.825030 15789 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:24:17.825214 15763 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:24:17.825299 15763 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786292417722232-21710-0/minicluster-data/ts-0/data,/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786292417722232-21710-0/minicluster-data/ts-0/wal
uuid: "0c28fcd823f4440aa2a17070dcdb3f19"
format_stamp: "Formatted at 2026-08-09 16:24:17 on dist-test-slave-s11t"
server_key: "3e45f0b02719e0215e4bd28e27db09e1"
server_key_iv: "3743e7069633f504a27f083f3c0ccc69"
server_key_version: "encryptionkey@0"
I20260809 16:24:17.825351 15763 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786292417722232-21710-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786292417722232-21710-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786292417722232-21710-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:24:17.837096 15763 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:24:17.839016 15763 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:24:17.839125 15763 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:24:17.839293 15763 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:24:17.839612 15763 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:24:17.839645 15763 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:24:17.839666 15763 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:24:17.839715 15763 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:24:17.845772 15763 rpc_server.cc:307] RPC server started. Bound to: 127.21.51.129:43933
I20260809 16:24:17.845858 15948 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.51.129:43933 every 8 connection(s)
I20260809 16:24:17.846163 15763 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786292417722232-21710-0/minicluster-data/ts-0/data/info.pb
I20260809 16:24:17.850976 21710 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrNqUCn/build/release/bin/kudu as pid 15763
I20260809 16:24:17.851104 21710 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786292417722232-21710-0/minicluster-data/ts-0/wal/instance
I20260809 16:24:17.851327 21710 external_mini_cluster.cc:1428] Setting key 146fda9a0d33ca0b7461f8a40df123cb
Aug 09 16:24:17 dist-test-slave-s11t krb5kdc[15596](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292657, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.51.129@KRBTEST.COM for kudu/127.21.51.190@KRBTEST.COM
I20260809 16:24:17.856017 15727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:17.846902 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.51.129:38755 (local address 127.21.51.190:39581)
0809 16:24:17.846984 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0809 16:24:17.846987 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 16:24:17.847175 (+ 188us) server_negotiation.cc:407] Connection header received
0809 16:24:17.847659 (+ 484us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:24:17.847660 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:24:17.847675 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:24:17.847736 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:24:17.848624 (+ 888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:24:17.849203 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:24:17.850170 (+ 967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:24:17.850394 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:24:17.850452 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:24:17.852816 (+ 2364us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:24:17.852817 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:24:17.852818 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:24:17.852825 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:24:17.854317 (+ 1492us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:24:17.854704 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:24:17.854705 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:24:17.854706 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:24:17.854925 (+ 219us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:24:17.855240 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:24:17.855241 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:24:17.855242 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:24:17.855426 (+ 184us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:24:17.855494 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0809 16:24:17.855796 (+ 302us) server_negotiation.cc:299] Negotiation successful
0809 16:24:17.855917 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260809 16:24:17.856199 15952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:17.846855 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:39581 (local address 127.21.51.129:38755)
0809 16:24:17.847026 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0809 16:24:17.847061 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0809 16:24:17.847563 (+ 502us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:24:17.847786 (+ 223us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:24:17.847789 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:24:17.848342 (+ 553us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:24:17.848537 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:24:17.848542 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:24:17.849292 (+ 750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:24:17.849293 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:24:17.850087 (+ 794us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:24:17.850089 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:24:17.850474 (+ 385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:24:17.850475 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:24:17.850571 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:24:17.851018 (+ 447us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:24:17.851028 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:24:17.852653 (+ 1625us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:24:17.854385 (+ 1732us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:24:17.854393 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:24:17.854396 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:24:17.854621 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:24:17.855016 (+ 395us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:24:17.855017 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:24:17.855017 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:24:17.855150 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:24:17.855498 (+ 348us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:24:17.855506 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:24:17.855651 (+ 145us) client_negotiation.cc:769] Sending connection context
0809 16:24:17.855772 (+ 121us) client_negotiation.cc:240] Negotiation successful
0809 16:24:17.855940 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":108,"threads_started":1}
I20260809 16:24:17.856715 15949 heartbeater.cc:347] Connected to a master server at 127.21.51.190:39581
I20260809 16:24:17.856817 15949 heartbeater.cc:464] Registering TS with master...
I20260809 16:24:17.857076 15949 heartbeater.cc:511] Master 127.21.51.190:39581 requested a full tablet report, sending...
I20260809 16:24:17.857625 15664 ts_manager.cc:194] Registered new tserver with Master: 0c28fcd823f4440aa2a17070dcdb3f19 (127.21.51.129:43933)
I20260809 16:24:17.858300 15664 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.51.129@KRBTEST.COM'} at 127.21.51.129:38755
WARNING: no policy specified for kudu/127.21.51.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.51.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.51.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.130.keytab.
Entry for principal kudu/127.21.51.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.130.keytab.
I20260809 16:24:17.879765 21710 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrNqUCn/build/release/bin/kudu
/tmp/dist-test-taskrNqUCn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786292417722232-21710-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786292417722232-21710-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786292417722232-21710-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786292417722232-21710-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.51.130:0
--local_ip_for_outbound_sockets=127.21.51.130
--webserver_interface=127.21.51.130
--webserver_port=0
--tserver_master_addrs=127.21.51.190:39581
--keytab_file=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.130.keytab
--principal=kudu/127.21.51.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.51.148:34019
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786292417722232-21710-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786292417722232-21710-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786292417722232-21710-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:24:17.962113 15959 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:24:17.962313 15959 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:24:17.962337 15959 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:24:17.963486 15959 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:24:17.963532 15959 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:24:17.963558 15959 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.51.130
I20260809 16:24:17.965044 15959 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.51.148:34019
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786292417722232-21710-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786292417722232-21710-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.130.keytab
--principal=kudu/127.21.51.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.51.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786292417722232-21710-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.51.130
--webserver_port=0
--tserver_master_addrs=127.21.51.190:39581
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15959
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.51.130
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786292417722232-21710-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:24:17.965289 15959 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:24:17.965528 15959 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:24:17.970590 15970 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:24:17.970652 15959 server_base.cc:1034] running on GCE node
W20260809 16:24:17.970618 15966 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:24:17.970618 15965 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:24:17.971066 15959 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:24:17.971316 15959 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:24:17.972496 15959 hybrid_clock.cc:648] HybridClock initialized: now 1786292657972484 us; error 31 us; skew 500 ppm
Aug 09 16:24:17 dist-test-slave-s11t krb5kdc[15596](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292657, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.51.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:24:17.975099 15959 init.cc:377] Logged in from keytab as kudu/127.21.51.130@KRBTEST.COM (short username kudu)
I20260809 16:24:17.975497 15959 webserver.cc:460] Webserver started at http://127.21.51.130:37495/ using document root <none> and password file <none>
I20260809 16:24:17.975678 15959 fs_manager.cc:359] Metadata directory not provided
I20260809 16:24:17.975718 15959 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:24:17.975809 15959 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:24:17.976882 15959 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786292417722232-21710-0/minicluster-data/ts-1/data/instance:
uuid: "2f553539ef724f5b931f9b778a8f37c0"
format_stamp: "Formatted at 2026-08-09 16:24:17 on dist-test-slave-s11t"
server_key: "ebe799e571325a05c4109c4f5eaf657e"
server_key_iv: "efdb1485e5dd0091bab4f8a213fc100f"
server_key_version: "encryptionkey@0"
I20260809 16:24:17.977195 15959 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786292417722232-21710-0/minicluster-data/ts-1/wal/instance:
uuid: "2f553539ef724f5b931f9b778a8f37c0"
format_stamp: "Formatted at 2026-08-09 16:24:17 on dist-test-slave-s11t"
server_key: "ebe799e571325a05c4109c4f5eaf657e"
server_key_iv: "efdb1485e5dd0091bab4f8a213fc100f"
server_key_version: "encryptionkey@0"
I20260809 16:24:17.978478 15959 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:24:17.979226 15979 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:24:17.979466 15959 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:24:17.979558 15959 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786292417722232-21710-0/minicluster-data/ts-1/data,/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786292417722232-21710-0/minicluster-data/ts-1/wal
uuid: "2f553539ef724f5b931f9b778a8f37c0"
format_stamp: "Formatted at 2026-08-09 16:24:17 on dist-test-slave-s11t"
server_key: "ebe799e571325a05c4109c4f5eaf657e"
server_key_iv: "efdb1485e5dd0091bab4f8a213fc100f"
server_key_version: "encryptionkey@0"
I20260809 16:24:17.979614 15959 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786292417722232-21710-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786292417722232-21710-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786292417722232-21710-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:24:18.000351 15959 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:24:18.002730 15959 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:24:18.002859 15959 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:24:18.003065 15959 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:24:18.003399 15959 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:24:18.003437 15959 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:24:18.003460 15959 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:24:18.003473 15959 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:24:18.009549 15959 rpc_server.cc:307] RPC server started. Bound to: 127.21.51.130:39693
I20260809 16:24:18.009630 16147 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.51.130:39693 every 8 connection(s)
I20260809 16:24:18.009975 15959 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786292417722232-21710-0/minicluster-data/ts-1/data/info.pb
I20260809 16:24:18.015553 21710 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrNqUCn/build/release/bin/kudu as pid 15959
I20260809 16:24:18.015679 21710 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786292417722232-21710-0/minicluster-data/ts-1/wal/instance
I20260809 16:24:18.015913 21710 external_mini_cluster.cc:1428] Setting key c1cdb3cf5b18702fee3ab66574854f54
Aug 09 16:24:18 dist-test-slave-s11t krb5kdc[15596](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292657, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.51.130@KRBTEST.COM for kudu/127.21.51.190@KRBTEST.COM
I20260809 16:24:18.020716 15727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:18.010938 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.51.130:55025 (local address 127.21.51.190:39581)
0809 16:24:18.011017 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0809 16:24:18.011019 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:24:18.011232 (+ 213us) server_negotiation.cc:407] Connection header received
0809 16:24:18.011902 (+ 670us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:24:18.011903 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:24:18.011914 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:24:18.011973 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:24:18.013005 (+ 1032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:24:18.013521 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:24:18.014492 (+ 971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:24:18.014685 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:24:18.014795 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:24:18.017424 (+ 2629us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:24:18.017425 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:24:18.017426 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:24:18.017432 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:24:18.019288 (+ 1856us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:24:18.019674 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:24:18.019675 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:24:18.019676 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:24:18.019757 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:24:18.020000 (+ 243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:24:18.020001 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:24:18.020002 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:24:18.020146 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:24:18.020220 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0809 16:24:18.020456 (+ 236us) server_negotiation.cc:299] Negotiation successful
0809 16:24:18.020552 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260809 16:24:18.020735 16155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:18.010912 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:39581 (local address 127.21.51.130:55025)
0809 16:24:18.011095 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0809 16:24:18.011138 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0809 16:24:18.011763 (+ 625us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:24:18.012030 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:24:18.012033 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:24:18.012618 (+ 585us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:24:18.012904 (+ 286us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:24:18.012909 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:24:18.013604 (+ 695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:24:18.013605 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:24:18.014402 (+ 797us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:24:18.014404 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:24:18.014817 (+ 413us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:24:18.014818 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:24:18.014930 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:24:18.015460 (+ 530us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:24:18.015470 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:24:18.017335 (+ 1865us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:24:18.019371 (+ 2036us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:24:18.019378 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:24:18.019380 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:24:18.019596 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:24:18.019829 (+ 233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:24:18.019830 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:24:18.019830 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:24:18.019932 (+ 102us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:24:18.020213 (+ 281us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:24:18.020217 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:24:18.020326 (+ 109us) client_negotiation.cc:769] Sending connection context
0809 16:24:18.020409 (+ 83us) client_negotiation.cc:240] Negotiation successful
0809 16:24:18.020528 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":99,"threads_started":1}
I20260809 16:24:18.021319 16150 heartbeater.cc:347] Connected to a master server at 127.21.51.190:39581
I20260809 16:24:18.021425 16150 heartbeater.cc:464] Registering TS with master...
I20260809 16:24:18.021603 16150 heartbeater.cc:511] Master 127.21.51.190:39581 requested a full tablet report, sending...
I20260809 16:24:18.022043 15664 ts_manager.cc:194] Registered new tserver with Master: 2f553539ef724f5b931f9b778a8f37c0 (127.21.51.130:39693)
I20260809 16:24:18.022578 15664 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.51.130@KRBTEST.COM'} at 127.21.51.130:55025
WARNING: no policy specified for kudu/127.21.51.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.51.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.51.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.131.keytab.
Entry for principal kudu/127.21.51.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.131.keytab.
I20260809 16:24:18.045686 21710 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrNqUCn/build/release/bin/kudu
/tmp/dist-test-taskrNqUCn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786292417722232-21710-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786292417722232-21710-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786292417722232-21710-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786292417722232-21710-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.51.131:0
--local_ip_for_outbound_sockets=127.21.51.131
--webserver_interface=127.21.51.131
--webserver_port=0
--tserver_master_addrs=127.21.51.190:39581
--keytab_file=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.131.keytab
--principal=kudu/127.21.51.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.51.148:34019
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786292417722232-21710-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786292417722232-21710-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786292417722232-21710-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:24:18.128830 16159 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:24:18.129021 16159 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:24:18.129045 16159 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:24:18.130107 16159 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:24:18.130146 16159 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:24:18.130172 16159 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.51.131
I20260809 16:24:18.131619 16159 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.51.148:34019
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786292417722232-21710-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786292417722232-21710-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.131.keytab
--principal=kudu/127.21.51.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.51.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786292417722232-21710-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.51.131
--webserver_port=0
--tserver_master_addrs=127.21.51.190:39581
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16159
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.51.131
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786292417722232-21710-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:24:18.131798 16159 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:24:18.131979 16159 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:24:18.136909 16168 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:24:18.136982 16166 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:24:18.137071 16159 server_base.cc:1034] running on GCE node
W20260809 16:24:18.137099 16165 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:24:18.137451 16159 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:24:18.137661 16159 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:24:18.138835 16159 hybrid_clock.cc:648] HybridClock initialized: now 1786292658138823 us; error 30 us; skew 500 ppm
Aug 09 16:24:18 dist-test-slave-s11t krb5kdc[15596](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292658, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.51.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:24:18.141454 16159 init.cc:377] Logged in from keytab as kudu/127.21.51.131@KRBTEST.COM (short username kudu)
I20260809 16:24:18.141883 16159 webserver.cc:460] Webserver started at http://127.21.51.131:33735/ using document root <none> and password file <none>
I20260809 16:24:18.142062 16159 fs_manager.cc:359] Metadata directory not provided
I20260809 16:24:18.142102 16159 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:24:18.142189 16159 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:24:18.143146 16159 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786292417722232-21710-0/minicluster-data/ts-2/data/instance:
uuid: "10021826a9bb457b931262ed655abf8c"
format_stamp: "Formatted at 2026-08-09 16:24:18 on dist-test-slave-s11t"
server_key: "ba1ee14ab838a4c77badc014a67a0a10"
server_key_iv: "bd161b35d7c0b46f3affc1afc1f7f7b6"
server_key_version: "encryptionkey@0"
I20260809 16:24:18.143445 16159 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786292417722232-21710-0/minicluster-data/ts-2/wal/instance:
uuid: "10021826a9bb457b931262ed655abf8c"
format_stamp: "Formatted at 2026-08-09 16:24:18 on dist-test-slave-s11t"
server_key: "ba1ee14ab838a4c77badc014a67a0a10"
server_key_iv: "bd161b35d7c0b46f3affc1afc1f7f7b6"
server_key_version: "encryptionkey@0"
I20260809 16:24:18.144738 16159 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260809 16:24:18.145440 16178 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:24:18.145624 16159 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 16:24:18.145717 16159 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786292417722232-21710-0/minicluster-data/ts-2/data,/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786292417722232-21710-0/minicluster-data/ts-2/wal
uuid: "10021826a9bb457b931262ed655abf8c"
format_stamp: "Formatted at 2026-08-09 16:24:18 on dist-test-slave-s11t"
server_key: "ba1ee14ab838a4c77badc014a67a0a10"
server_key_iv: "bd161b35d7c0b46f3affc1afc1f7f7b6"
server_key_version: "encryptionkey@0"
I20260809 16:24:18.145790 16159 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786292417722232-21710-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786292417722232-21710-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786292417722232-21710-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:24:18.154990 16159 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:24:18.157048 16159 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:24:18.157208 16159 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:24:18.157433 16159 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:24:18.157742 16159 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:24:18.157796 16159 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:24:18.157836 16159 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:24:18.157862 16159 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:24:18.163614 16159 rpc_server.cc:307] RPC server started. Bound to: 127.21.51.131:34159
I20260809 16:24:18.163684 16332 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.51.131:34159 every 8 connection(s)
I20260809 16:24:18.164014 16159 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786292417722232-21710-0/minicluster-data/ts-2/data/info.pb
Aug 09 16:24:18 dist-test-slave-s11t krb5kdc[15596](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292658, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.51.131@KRBTEST.COM for kudu/127.21.51.190@KRBTEST.COM
I20260809 16:24:18.171178 21710 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrNqUCn/build/release/bin/kudu as pid 16159
I20260809 16:24:18.171290 21710 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786292417722232-21710-0/minicluster-data/ts-2/wal/instance
I20260809 16:24:18.171515 21710 external_mini_cluster.cc:1428] Setting key 9034cb6092128eed5187ea3e8c50203a
I20260809 16:24:18.173448 15727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:18.164775 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.51.131:60715 (local address 127.21.51.190:39581)
0809 16:24:18.164820 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0809 16:24:18.164822 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:24:18.165030 (+ 208us) server_negotiation.cc:407] Connection header received
0809 16:24:18.165554 (+ 524us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:24:18.165556 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:24:18.165568 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:24:18.165638 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:24:18.166609 (+ 971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:24:18.167141 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:24:18.168148 (+ 1007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:24:18.168346 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:24:18.168454 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:24:18.170616 (+ 2162us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:24:18.170618 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:24:18.170618 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:24:18.170626 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:24:18.172043 (+ 1417us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:24:18.172453 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:24:18.172455 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:24:18.172456 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:24:18.172514 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:24:18.172804 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:24:18.172805 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:24:18.172805 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:24:18.172930 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:24:18.173008 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0809 16:24:18.173278 (+ 270us) server_negotiation.cc:299] Negotiation successful
0809 16:24:18.173360 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260809 16:24:18.173640 16336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:18.164731 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:39581 (local address 127.21.51.131:60715)
0809 16:24:18.164897 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0809 16:24:18.164942 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0809 16:24:18.165449 (+ 507us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:24:18.165707 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:24:18.165711 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:24:18.166303 (+ 592us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:24:18.166505 (+ 202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:24:18.166509 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:24:18.167244 (+ 735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:24:18.167246 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:24:18.168055 (+ 809us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:24:18.168057 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:24:18.168461 (+ 404us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:24:18.168462 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:24:18.168594 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:24:18.169114 (+ 520us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:24:18.169122 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:24:18.170518 (+ 1396us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:24:18.172130 (+ 1612us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:24:18.172136 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:24:18.172138 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:24:18.172356 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:24:18.172580 (+ 224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:24:18.172581 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:24:18.172581 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:24:18.172738 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:24:18.172998 (+ 260us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:24:18.173005 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:24:18.173142 (+ 137us) client_negotiation.cc:769] Sending connection context
0809 16:24:18.173240 (+ 98us) client_negotiation.cc:240] Negotiation successful
0809 16:24:18.173397 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":71,"threads_started":1}
I20260809 16:24:18.174088 16333 heartbeater.cc:347] Connected to a master server at 127.21.51.190:39581
I20260809 16:24:18.174170 16333 heartbeater.cc:464] Registering TS with master...
I20260809 16:24:18.174301 16333 heartbeater.cc:511] Master 127.21.51.190:39581 requested a full tablet report, sending...
I20260809 16:24:18.174602 15664 ts_manager.cc:194] Registered new tserver with Master: 10021826a9bb457b931262ed655abf8c (127.21.51.131:34159)
I20260809 16:24:18.175016 15664 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.51.131@KRBTEST.COM'} at 127.21.51.131:60715
I20260809 16:24:18.183368 21710 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 16:24:18.190765 15727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:18.184640 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45618 (local address 127.21.51.190:39581)
0809 16:24:18.184754 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0809 16:24:18.184756 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:24:18.184845 (+ 89us) server_negotiation.cc:407] Connection header received
0809 16:24:18.184907 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:24:18.184908 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:24:18.184918 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:24:18.184974 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:24:18.185548 (+ 574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:24:18.186040 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:24:18.186777 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:24:18.186996 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:24:18.187073 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:24:18.188284 (+ 1211us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:24:18.188286 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:24:18.188286 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:24:18.188294 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:24:18.189581 (+ 1287us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:24:18.189935 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:24:18.189937 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:24:18.189937 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:24:18.189999 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:24:18.190210 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:24:18.190211 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:24:18.190211 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:24:18.190367 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:24:18.190480 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0809 16:24:18.190579 (+ 99us) server_negotiation.cc:299] Negotiation successful
0809 16:24:18.190664 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260809 16:24:18.192517 15664 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45618:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 16:24:18.194139 15664 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 16:24:18.200476 16363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:18.196429 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.130:39693 (local address 127.0.0.1:40462)
0809 16:24:18.196782 (+ 353us) negotiation.cc:107] Waiting for socket to connect
0809 16:24:18.196827 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0809 16:24:18.196904 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:24:18.197208 (+ 304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:24:18.197209 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:24:18.197210 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:24:18.197262 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:24:18.197264 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:24:18.198224 (+ 960us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:24:18.198225 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:24:18.199341 (+ 1116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:24:18.199344 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:24:18.200192 (+ 848us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:24:18.200193 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:24:18.200258 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:24:18.200267 (+ 9us) client_negotiation.cc:769] Sending connection context
0809 16:24:18.200349 (+ 82us) client_negotiation.cc:240] Negotiation successful
0809 16:24:18.200403 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":320,"thread_start_us":47,"threads_started":1}
I20260809 16:24:18.200488 16361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:18.196533 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40462 (local address 127.21.51.130:39693)
0809 16:24:18.196789 (+ 256us) server_negotiation.cc:206] Beginning negotiation
0809 16:24:18.196794 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 16:24:18.196915 (+ 121us) server_negotiation.cc:407] Connection header received
0809 16:24:18.197031 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:24:18.197033 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:24:18.197051 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:24:18.197106 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:24:18.197355 (+ 249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:24:18.198121 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:24:18.199440 (+ 1319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:24:18.200070 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:24:18.200105 (+ 35us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:24:18.200143 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0809 16:24:18.200368 (+ 225us) server_negotiation.cc:299] Negotiation successful
0809 16:24:18.200410 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":221,"thread_start_us":76,"threads_started":1}
I20260809 16:24:18.200744 16359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:18.196622 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.131:34159 (local address 127.0.0.1:44816)
0809 16:24:18.197008 (+ 386us) negotiation.cc:107] Waiting for socket to connect
0809 16:24:18.197017 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 16:24:18.197065 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:24:18.197332 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:24:18.197334 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:24:18.197337 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:24:18.197395 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:24:18.197397 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:24:18.198662 (+ 1265us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:24:18.198663 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:24:18.199524 (+ 861us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:24:18.199527 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:24:18.200477 (+ 950us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:24:18.200478 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:24:18.200541 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:24:18.200545 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 16:24:18.200597 (+ 52us) client_negotiation.cc:240] Negotiation successful
0809 16:24:18.200639 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":377,"thread_start_us":95,"threads_started":1}
I20260809 16:24:18.200939 16364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:18.196786 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44816 (local address 127.21.51.131:34159)
0809 16:24:18.197007 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0809 16:24:18.197012 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 16:24:18.197057 (+ 45us) server_negotiation.cc:407] Connection header received
0809 16:24:18.197148 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:24:18.197150 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:24:18.197165 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:24:18.197248 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:24:18.197481 (+ 233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:24:18.198552 (+ 1071us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:24:18.199626 (+ 1074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:24:18.200314 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:24:18.200387 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:24:18.200438 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0809 16:24:18.200812 (+ 374us) server_negotiation.cc:299] Negotiation successful
0809 16:24:18.200863 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":184,"thread_start_us":100,"threads_started":1}
I20260809 16:24:18.201190 16362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:18.196570 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42812 (local address 127.21.51.129:43933)
0809 16:24:18.197019 (+ 449us) server_negotiation.cc:206] Beginning negotiation
0809 16:24:18.197022 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 16:24:18.197032 (+ 10us) server_negotiation.cc:407] Connection header received
0809 16:24:18.197072 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:24:18.197074 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:24:18.197090 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:24:18.197192 (+ 102us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:24:18.197389 (+ 197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:24:18.198523 (+ 1134us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:24:18.199844 (+ 1321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:24:18.200542 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:24:18.200606 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:24:18.200652 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0809 16:24:18.201038 (+ 386us) server_negotiation.cc:299] Negotiation successful
0809 16:24:18.201087 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":413,"thread_start_us":91,"threads_started":1}
I20260809 16:24:18.200970 16360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:18.196449 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.129:43933 (local address 127.0.0.1:42812)
0809 16:24:18.196838 (+ 389us) negotiation.cc:107] Waiting for socket to connect
0809 16:24:18.196849 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 16:24:18.196904 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:24:18.197192 (+ 288us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:24:18.197196 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:24:18.197198 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:24:18.197262 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:24:18.197267 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:24:18.198647 (+ 1380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:24:18.198648 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:24:18.199756 (+ 1108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:24:18.199759 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:24:18.200731 (+ 972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:24:18.200732 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:24:18.200809 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:24:18.200813 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 16:24:18.200867 (+ 54us) client_negotiation.cc:240] Negotiation successful
0809 16:24:18.200903 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":372,"mutex_wait_us":8,"thread_start_us":105,"threads_started":1}
I20260809 16:24:18.201638 16051 tablet_service.cc:1467] Processing CreateTablet for tablet 5e84c1913f544a479fdffb4059f7fd54 (DEFAULT_TABLE table=test-table [id=732fbcec0f6548498596d54537ab616d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:24:18.201737 16242 tablet_service.cc:1467] Processing CreateTablet for tablet 5e84c1913f544a479fdffb4059f7fd54 (DEFAULT_TABLE table=test-table [id=732fbcec0f6548498596d54537ab616d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:24:18.201921 15852 tablet_service.cc:1467] Processing CreateTablet for tablet 5e84c1913f544a479fdffb4059f7fd54 (DEFAULT_TABLE table=test-table [id=732fbcec0f6548498596d54537ab616d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:24:18.202019 16242 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5e84c1913f544a479fdffb4059f7fd54. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:24:18.202019 16051 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5e84c1913f544a479fdffb4059f7fd54. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:24:18.202179 15852 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5e84c1913f544a479fdffb4059f7fd54. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:24:18.205338 16365 tablet_bootstrap.cc:492] T 5e84c1913f544a479fdffb4059f7fd54 P 0c28fcd823f4440aa2a17070dcdb3f19: Bootstrap starting.
I20260809 16:24:18.205818 16369 tablet_bootstrap.cc:492] T 5e84c1913f544a479fdffb4059f7fd54 P 2f553539ef724f5b931f9b778a8f37c0: Bootstrap starting.
I20260809 16:24:18.205981 16365 tablet_bootstrap.cc:654] T 5e84c1913f544a479fdffb4059f7fd54 P 0c28fcd823f4440aa2a17070dcdb3f19: Neither blocks nor log segments found. Creating new log.
I20260809 16:24:18.206416 16369 tablet_bootstrap.cc:654] T 5e84c1913f544a479fdffb4059f7fd54 P 2f553539ef724f5b931f9b778a8f37c0: Neither blocks nor log segments found. Creating new log.
I20260809 16:24:18.206521 16365 log.cc:824] T 5e84c1913f544a479fdffb4059f7fd54 P 0c28fcd823f4440aa2a17070dcdb3f19: Log is configured to *not* fsync() on all Append() calls
I20260809 16:24:18.207022 16369 log.cc:824] T 5e84c1913f544a479fdffb4059f7fd54 P 2f553539ef724f5b931f9b778a8f37c0: Log is configured to *not* fsync() on all Append() calls
I20260809 16:24:18.207334 16368 tablet_bootstrap.cc:492] T 5e84c1913f544a479fdffb4059f7fd54 P 10021826a9bb457b931262ed655abf8c: Bootstrap starting.
I20260809 16:24:18.207502 16365 tablet_bootstrap.cc:492] T 5e84c1913f544a479fdffb4059f7fd54 P 0c28fcd823f4440aa2a17070dcdb3f19: No bootstrap required, opened a new log
I20260809 16:24:18.207584 16365 ts_tablet_manager.cc:1397] T 5e84c1913f544a479fdffb4059f7fd54 P 0c28fcd823f4440aa2a17070dcdb3f19: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 16:24:18.207782 16369 tablet_bootstrap.cc:492] T 5e84c1913f544a479fdffb4059f7fd54 P 2f553539ef724f5b931f9b778a8f37c0: No bootstrap required, opened a new log
I20260809 16:24:18.207890 16369 ts_tablet_manager.cc:1397] T 5e84c1913f544a479fdffb4059f7fd54 P 2f553539ef724f5b931f9b778a8f37c0: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 16:24:18.208003 16368 tablet_bootstrap.cc:654] T 5e84c1913f544a479fdffb4059f7fd54 P 10021826a9bb457b931262ed655abf8c: Neither blocks nor log segments found. Creating new log.
I20260809 16:24:18.208498 16368 log.cc:824] T 5e84c1913f544a479fdffb4059f7fd54 P 10021826a9bb457b931262ed655abf8c: Log is configured to *not* fsync() on all Append() calls
I20260809 16:24:18.209215 16368 tablet_bootstrap.cc:492] T 5e84c1913f544a479fdffb4059f7fd54 P 10021826a9bb457b931262ed655abf8c: No bootstrap required, opened a new log
I20260809 16:24:18.209299 16368 ts_tablet_manager.cc:1397] T 5e84c1913f544a479fdffb4059f7fd54 P 10021826a9bb457b931262ed655abf8c: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260809 16:24:18.209913 16365 raft_consensus.cc:348] T 5e84c1913f544a479fdffb4059f7fd54 P 0c28fcd823f4440aa2a17070dcdb3f19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f553539ef724f5b931f9b778a8f37c0" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 39693 } } peers { permanent_uuid: "0c28fcd823f4440aa2a17070dcdb3f19" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 43933 } } peers { permanent_uuid: "10021826a9bb457b931262ed655abf8c" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 34159 } }
I20260809 16:24:18.209913 16369 raft_consensus.cc:348] T 5e84c1913f544a479fdffb4059f7fd54 P 2f553539ef724f5b931f9b778a8f37c0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f553539ef724f5b931f9b778a8f37c0" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 39693 } } peers { permanent_uuid: "0c28fcd823f4440aa2a17070dcdb3f19" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 43933 } } peers { permanent_uuid: "10021826a9bb457b931262ed655abf8c" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 34159 } }
I20260809 16:24:18.210104 16369 raft_consensus.cc:374] T 5e84c1913f544a479fdffb4059f7fd54 P 2f553539ef724f5b931f9b778a8f37c0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:24:18.210104 16365 raft_consensus.cc:374] T 5e84c1913f544a479fdffb4059f7fd54 P 0c28fcd823f4440aa2a17070dcdb3f19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:24:18.210144 16369 raft_consensus.cc:729] T 5e84c1913f544a479fdffb4059f7fd54 P 2f553539ef724f5b931f9b778a8f37c0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2f553539ef724f5b931f9b778a8f37c0, State: Initialized, Role: FOLLOWER
I20260809 16:24:18.210144 16365 raft_consensus.cc:729] T 5e84c1913f544a479fdffb4059f7fd54 P 0c28fcd823f4440aa2a17070dcdb3f19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0c28fcd823f4440aa2a17070dcdb3f19, State: Initialized, Role: FOLLOWER
I20260809 16:24:18.210290 16369 consensus_queue.cc:260] T 5e84c1913f544a479fdffb4059f7fd54 P 2f553539ef724f5b931f9b778a8f37c0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f553539ef724f5b931f9b778a8f37c0" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 39693 } } peers { permanent_uuid: "0c28fcd823f4440aa2a17070dcdb3f19" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 43933 } } peers { permanent_uuid: "10021826a9bb457b931262ed655abf8c" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 34159 } }
I20260809 16:24:18.210347 16365 consensus_queue.cc:260] T 5e84c1913f544a479fdffb4059f7fd54 P 0c28fcd823f4440aa2a17070dcdb3f19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f553539ef724f5b931f9b778a8f37c0" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 39693 } } peers { permanent_uuid: "0c28fcd823f4440aa2a17070dcdb3f19" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 43933 } } peers { permanent_uuid: "10021826a9bb457b931262ed655abf8c" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 34159 } }
I20260809 16:24:18.210597 16365 ts_tablet_manager.cc:1428] T 5e84c1913f544a479fdffb4059f7fd54 P 0c28fcd823f4440aa2a17070dcdb3f19: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 16:24:18.210597 16369 ts_tablet_manager.cc:1428] T 5e84c1913f544a479fdffb4059f7fd54 P 2f553539ef724f5b931f9b778a8f37c0: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 16:24:18.210644 15949 heartbeater.cc:503] Master 127.21.51.190:39581 was elected leader, sending a full tablet report...
I20260809 16:24:18.210711 16150 heartbeater.cc:503] Master 127.21.51.190:39581 was elected leader, sending a full tablet report...
I20260809 16:24:18.210970 16368 raft_consensus.cc:348] T 5e84c1913f544a479fdffb4059f7fd54 P 10021826a9bb457b931262ed655abf8c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f553539ef724f5b931f9b778a8f37c0" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 39693 } } peers { permanent_uuid: "0c28fcd823f4440aa2a17070dcdb3f19" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 43933 } } peers { permanent_uuid: "10021826a9bb457b931262ed655abf8c" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 34159 } }
I20260809 16:24:18.211143 16368 raft_consensus.cc:374] T 5e84c1913f544a479fdffb4059f7fd54 P 10021826a9bb457b931262ed655abf8c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:24:18.211186 16368 raft_consensus.cc:729] T 5e84c1913f544a479fdffb4059f7fd54 P 10021826a9bb457b931262ed655abf8c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 10021826a9bb457b931262ed655abf8c, State: Initialized, Role: FOLLOWER
I20260809 16:24:18.211305 16368 consensus_queue.cc:260] T 5e84c1913f544a479fdffb4059f7fd54 P 10021826a9bb457b931262ed655abf8c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f553539ef724f5b931f9b778a8f37c0" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 39693 } } peers { permanent_uuid: "0c28fcd823f4440aa2a17070dcdb3f19" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 43933 } } peers { permanent_uuid: "10021826a9bb457b931262ed655abf8c" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 34159 } }
I20260809 16:24:18.211578 16368 ts_tablet_manager.cc:1428] T 5e84c1913f544a479fdffb4059f7fd54 P 10021826a9bb457b931262ed655abf8c: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260809 16:24:18.211809 16333 heartbeater.cc:503] Master 127.21.51.190:39581 was elected leader, sending a full tablet report...
W20260809 16:24:18.260655 16152 tablet.cc:2369] T 5e84c1913f544a479fdffb4059f7fd54 P 2f553539ef724f5b931f9b778a8f37c0: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 16:24:18.346999 15950 tablet.cc:2369] T 5e84c1913f544a479fdffb4059f7fd54 P 0c28fcd823f4440aa2a17070dcdb3f19: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 16:24:18.354018 16373 raft_consensus.cc:482] T 5e84c1913f544a479fdffb4059f7fd54 P 2f553539ef724f5b931f9b778a8f37c0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 16:24:18.354158 16373 raft_consensus.cc:504] T 5e84c1913f544a479fdffb4059f7fd54 P 2f553539ef724f5b931f9b778a8f37c0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f553539ef724f5b931f9b778a8f37c0" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 39693 } } peers { permanent_uuid: "0c28fcd823f4440aa2a17070dcdb3f19" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 43933 } } peers { permanent_uuid: "10021826a9bb457b931262ed655abf8c" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 34159 } }
I20260809 16:24:18.354588 16373 leader_election.cc:288] T 5e84c1913f544a479fdffb4059f7fd54 P 2f553539ef724f5b931f9b778a8f37c0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0c28fcd823f4440aa2a17070dcdb3f19 (127.21.51.129:43933), 10021826a9bb457b931262ed655abf8c (127.21.51.131:34159)
I20260809 16:24:18.357882 16155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:18.354797 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.131:34159 (local address 127.21.51.130:41751)
0809 16:24:18.354856 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0809 16:24:18.354869 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 16:24:18.354915 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:24:18.355095 (+ 180us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:24:18.355096 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:24:18.355098 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:24:18.355182 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:24:18.355184 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:24:18.356082 (+ 898us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:24:18.356083 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:24:18.356955 (+ 872us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:24:18.356957 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:24:18.357628 (+ 671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:24:18.357629 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:24:18.357717 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:24:18.357722 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 16:24:18.357778 (+ 56us) client_negotiation.cc:240] Negotiation successful
0809 16:24:18.357826 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40,"mutex_wait_us":18}
I20260809 16:24:18.357932 16364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:18.354865 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.51.130:41751 (local address 127.21.51.131:34159)
0809 16:24:18.354910 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0809 16:24:18.354912 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:24:18.354925 (+ 13us) server_negotiation.cc:407] Connection header received
0809 16:24:18.354964 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:24:18.354965 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:24:18.354976 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:24:18.355055 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:24:18.355304 (+ 249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:24:18.355976 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:24:18.357041 (+ 1065us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:24:18.357544 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:24:18.357607 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:24:18.357615 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0809 16:24:18.357809 (+ 194us) server_negotiation.cc:299] Negotiation successful
0809 16:24:18.357853 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260809 16:24:18.358114 16362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:18.354851 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.51.130:44817 (local address 127.21.51.129:43933)
0809 16:24:18.354916 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0809 16:24:18.354918 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:24:18.355111 (+ 193us) server_negotiation.cc:407] Connection header received
0809 16:24:18.355147 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:24:18.355149 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:24:18.355160 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:24:18.355224 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:24:18.355490 (+ 266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:24:18.355980 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:24:18.357268 (+ 1288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:24:18.357654 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:24:18.357726 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:24:18.357735 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0809 16:24:18.357997 (+ 262us) server_negotiation.cc:299] Negotiation successful
0809 16:24:18.358045 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260809 16:24:18.358230 16272 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e84c1913f544a479fdffb4059f7fd54" candidate_uuid: "2f553539ef724f5b931f9b778a8f37c0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "10021826a9bb457b931262ed655abf8c" is_pre_election: true
I20260809 16:24:18.358355 16376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:18.354797 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.129:43933 (local address 127.21.51.130:44817)
0809 16:24:18.355027 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0809 16:24:18.355038 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 16:24:18.355100 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:24:18.355305 (+ 205us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:24:18.355306 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:24:18.355309 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:24:18.355386 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:24:18.355388 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:24:18.356095 (+ 707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:24:18.356096 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:24:18.357173 (+ 1077us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:24:18.357176 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:24:18.357810 (+ 634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:24:18.357811 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:24:18.357886 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:24:18.357890 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 16:24:18.358240 (+ 350us) client_negotiation.cc:240] Negotiation successful
0809 16:24:18.358283 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":210,"mutex_wait_us":19,"thread_start_us":145,"threads_started":1}
I20260809 16:24:18.358383 16272 raft_consensus.cc:2454] T 5e84c1913f544a479fdffb4059f7fd54 P 10021826a9bb457b931262ed655abf8c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2f553539ef724f5b931f9b778a8f37c0 in term 0.
I20260809 16:24:18.358603 15980 leader_election.cc:302] T 5e84c1913f544a479fdffb4059f7fd54 P 2f553539ef724f5b931f9b778a8f37c0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10021826a9bb457b931262ed655abf8c, 2f553539ef724f5b931f9b778a8f37c0; no voters:
I20260809 16:24:18.358740 16373 raft_consensus.cc:2781] T 5e84c1913f544a479fdffb4059f7fd54 P 2f553539ef724f5b931f9b778a8f37c0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 16:24:18.358721 15884 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e84c1913f544a479fdffb4059f7fd54" candidate_uuid: "2f553539ef724f5b931f9b778a8f37c0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0c28fcd823f4440aa2a17070dcdb3f19" is_pre_election: true
I20260809 16:24:18.358804 16373 raft_consensus.cc:482] T 5e84c1913f544a479fdffb4059f7fd54 P 2f553539ef724f5b931f9b778a8f37c0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 16:24:18.358829 16373 raft_consensus.cc:3037] T 5e84c1913f544a479fdffb4059f7fd54 P 2f553539ef724f5b931f9b778a8f37c0 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:24:18.358870 15884 raft_consensus.cc:2454] T 5e84c1913f544a479fdffb4059f7fd54 P 0c28fcd823f4440aa2a17070dcdb3f19 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2f553539ef724f5b931f9b778a8f37c0 in term 0.
I20260809 16:24:18.359560 16373 raft_consensus.cc:504] T 5e84c1913f544a479fdffb4059f7fd54 P 2f553539ef724f5b931f9b778a8f37c0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f553539ef724f5b931f9b778a8f37c0" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 39693 } } peers { permanent_uuid: "0c28fcd823f4440aa2a17070dcdb3f19" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 43933 } } peers { permanent_uuid: "10021826a9bb457b931262ed655abf8c" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 34159 } }
I20260809 16:24:18.359695 16373 leader_election.cc:288] T 5e84c1913f544a479fdffb4059f7fd54 P 2f553539ef724f5b931f9b778a8f37c0 [CANDIDATE]: Term 1 election: Requested vote from peers 0c28fcd823f4440aa2a17070dcdb3f19 (127.21.51.129:43933), 10021826a9bb457b931262ed655abf8c (127.21.51.131:34159)
I20260809 16:24:18.359874 15884 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e84c1913f544a479fdffb4059f7fd54" candidate_uuid: "2f553539ef724f5b931f9b778a8f37c0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0c28fcd823f4440aa2a17070dcdb3f19"
I20260809 16:24:18.359861 16272 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e84c1913f544a479fdffb4059f7fd54" candidate_uuid: "2f553539ef724f5b931f9b778a8f37c0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "10021826a9bb457b931262ed655abf8c"
I20260809 16:24:18.359943 16272 raft_consensus.cc:3037] T 5e84c1913f544a479fdffb4059f7fd54 P 10021826a9bb457b931262ed655abf8c [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:24:18.359965 15884 raft_consensus.cc:3037] T 5e84c1913f544a479fdffb4059f7fd54 P 0c28fcd823f4440aa2a17070dcdb3f19 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:24:18.360649 15884 raft_consensus.cc:2454] T 5e84c1913f544a479fdffb4059f7fd54 P 0c28fcd823f4440aa2a17070dcdb3f19 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2f553539ef724f5b931f9b778a8f37c0 in term 1.
I20260809 16:24:18.360746 16272 raft_consensus.cc:2454] T 5e84c1913f544a479fdffb4059f7fd54 P 10021826a9bb457b931262ed655abf8c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2f553539ef724f5b931f9b778a8f37c0 in term 1.
I20260809 16:24:18.360857 15982 leader_election.cc:302] T 5e84c1913f544a479fdffb4059f7fd54 P 2f553539ef724f5b931f9b778a8f37c0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0c28fcd823f4440aa2a17070dcdb3f19, 2f553539ef724f5b931f9b778a8f37c0; no voters:
I20260809 16:24:18.360949 16373 raft_consensus.cc:2781] T 5e84c1913f544a479fdffb4059f7fd54 P 2f553539ef724f5b931f9b778a8f37c0 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:24:18.361081 16373 raft_consensus.cc:686] T 5e84c1913f544a479fdffb4059f7fd54 P 2f553539ef724f5b931f9b778a8f37c0 [term 1 LEADER]: Becoming Leader. State: Replica: 2f553539ef724f5b931f9b778a8f37c0, State: Running, Role: LEADER
I20260809 16:24:18.361186 16373 consensus_queue.cc:237] T 5e84c1913f544a479fdffb4059f7fd54 P 2f553539ef724f5b931f9b778a8f37c0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f553539ef724f5b931f9b778a8f37c0" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 39693 } } peers { permanent_uuid: "0c28fcd823f4440aa2a17070dcdb3f19" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 43933 } } peers { permanent_uuid: "10021826a9bb457b931262ed655abf8c" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 34159 } }
I20260809 16:24:18.361994 15662 catalog_manager.cc:5495] T 5e84c1913f544a479fdffb4059f7fd54 P 2f553539ef724f5b931f9b778a8f37c0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2f553539ef724f5b931f9b778a8f37c0 (127.21.51.130). New cstate: current_term: 1 leader_uuid: "2f553539ef724f5b931f9b778a8f37c0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f553539ef724f5b931f9b778a8f37c0" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 39693 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0c28fcd823f4440aa2a17070dcdb3f19" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 43933 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "10021826a9bb457b931262ed655abf8c" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 34159 } health_report { overall_health: UNKNOWN } } }
I20260809 16:24:18.380808 16361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:18.378572 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40468 (local address 127.21.51.130:39693)
0809 16:24:18.378631 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0809 16:24:18.378632 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 16:24:18.378668 (+ 36us) server_negotiation.cc:407] Connection header received
0809 16:24:18.378705 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:24:18.378705 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:24:18.378721 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:24:18.378775 (+ 54us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 16:24:18.378906 (+ 131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:24:18.379367 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:24:18.380088 (+ 721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:24:18.380263 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:24:18.380311 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:24:18.380448 (+ 137us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 16:24:18.380543 (+ 95us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 16:24:18.380601 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0809 16:24:18.380645 (+ 44us) server_negotiation.cc:299] Negotiation successful
0809 16:24:18.380661 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260809 16:24:18.382623 15884 raft_consensus.cc:1261] T 5e84c1913f544a479fdffb4059f7fd54 P 0c28fcd823f4440aa2a17070dcdb3f19 [term 1 FOLLOWER]: Refusing update from remote peer 2f553539ef724f5b931f9b778a8f37c0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 16:24:18.382656 16272 raft_consensus.cc:1261] T 5e84c1913f544a479fdffb4059f7fd54 P 10021826a9bb457b931262ed655abf8c [term 1 FOLLOWER]: Refusing update from remote peer 2f553539ef724f5b931f9b778a8f37c0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 16:24:18.383018 16373 consensus_queue.cc:1035] T 5e84c1913f544a479fdffb4059f7fd54 P 2f553539ef724f5b931f9b778a8f37c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "10021826a9bb457b931262ed655abf8c" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 34159 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 16:24:18.383189 16377 consensus_queue.cc:1035] T 5e84c1913f544a479fdffb4059f7fd54 P 2f553539ef724f5b931f9b778a8f37c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0c28fcd823f4440aa2a17070dcdb3f19" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 43933 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 16:24:18.386925 16385 mvcc.cc:205] Tried to move back new op lower bound from 7316654728732368896 to 7316654728647925760. Current Snapshot: MvccSnapshot[applied={T|T < 7316654728732368896 or (T in {7316654728732368896})}]
I20260809 16:24:18.387442 16384 mvcc.cc:205] Tried to move back new op lower bound from 7316654728732368896 to 7316654728647925760. Current Snapshot: MvccSnapshot[applied={T|T < 7316654728732368896 or (T in {7316654728732368896})}]
I20260809 16:24:18.387712 15662 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45618:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 16:24:18.387804 15662 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45618:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 16:24:18.388583 15662 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 31245ca59a7c4c0485e7f0fe141a06ca: Sending DeleteTablet for 3 replicas of tablet 5e84c1913f544a479fdffb4059f7fd54
I20260809 16:24:18.388954 16242 tablet_service.cc:1514] Processing DeleteTablet for tablet 5e84c1913f544a479fdffb4059f7fd54 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:24:18 UTC) from {username='kudu', principal='kudu/127.21.51.190@KRBTEST.COM'} at 127.0.0.1:44816
I20260809 16:24:18.388953 16051 tablet_service.cc:1514] Processing DeleteTablet for tablet 5e84c1913f544a479fdffb4059f7fd54 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:24:18 UTC) from {username='kudu', principal='kudu/127.21.51.190@KRBTEST.COM'} at 127.0.0.1:40462
I20260809 16:24:18.389207 16396 tablet_replica.cc:330] T 5e84c1913f544a479fdffb4059f7fd54 P 2f553539ef724f5b931f9b778a8f37c0: stopping tablet replica
I20260809 16:24:18.389322 15852 tablet_service.cc:1514] Processing DeleteTablet for tablet 5e84c1913f544a479fdffb4059f7fd54 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:24:18 UTC) from {username='kudu', principal='kudu/127.21.51.190@KRBTEST.COM'} at 127.0.0.1:42812
I20260809 16:24:18.389349 16396 raft_consensus.cc:2229] T 5e84c1913f544a479fdffb4059f7fd54 P 2f553539ef724f5b931f9b778a8f37c0 [term 1 LEADER]: Raft consensus shutting down.
I20260809 16:24:18.389519 16396 raft_consensus.cc:2258] T 5e84c1913f544a479fdffb4059f7fd54 P 2f553539ef724f5b931f9b778a8f37c0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 16:24:18.389535 16400 tablet_replica.cc:330] T 5e84c1913f544a479fdffb4059f7fd54 P 0c28fcd823f4440aa2a17070dcdb3f19: stopping tablet replica
I20260809 16:24:18.389624 16400 raft_consensus.cc:2229] T 5e84c1913f544a479fdffb4059f7fd54 P 0c28fcd823f4440aa2a17070dcdb3f19 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 16:24:18.389739 16400 raft_consensus.cc:2258] T 5e84c1913f544a479fdffb4059f7fd54 P 0c28fcd823f4440aa2a17070dcdb3f19 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 16:24:18.390130 16396 ts_tablet_manager.cc:1905] T 5e84c1913f544a479fdffb4059f7fd54 P 2f553539ef724f5b931f9b778a8f37c0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 16:24:18.390226 16400 ts_tablet_manager.cc:1905] T 5e84c1913f544a479fdffb4059f7fd54 P 0c28fcd823f4440aa2a17070dcdb3f19: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 16:24:18.391234 21710 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrNqUCn/build/release/bin/kudu with pid 15763
I20260809 16:24:18.391970 16396 ts_tablet_manager.cc:1918] T 5e84c1913f544a479fdffb4059f7fd54 P 2f553539ef724f5b931f9b778a8f37c0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 16:24:18.392047 16396 log.cc:1196] T 5e84c1913f544a479fdffb4059f7fd54 P 2f553539ef724f5b931f9b778a8f37c0: Deleting WAL directory at /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786292417722232-21710-0/minicluster-data/ts-1/wal/wals/5e84c1913f544a479fdffb4059f7fd54
I20260809 16:24:18.392326 16396 ts_tablet_manager.cc:1939] T 5e84c1913f544a479fdffb4059f7fd54 P 2f553539ef724f5b931f9b778a8f37c0: Deleting consensus metadata
I20260809 16:24:18.392762 16398 tablet_replica.cc:330] T 5e84c1913f544a479fdffb4059f7fd54 P 10021826a9bb457b931262ed655abf8c: stopping tablet replica
I20260809 16:24:18.392783 15635 catalog_manager.cc:4841] TS 2f553539ef724f5b931f9b778a8f37c0 (127.21.51.130:39693): tablet 5e84c1913f544a479fdffb4059f7fd54 (table test-table [id=732fbcec0f6548498596d54537ab616d]) successfully deleted
I20260809 16:24:18.392860 16398 raft_consensus.cc:2229] T 5e84c1913f544a479fdffb4059f7fd54 P 10021826a9bb457b931262ed655abf8c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 16:24:18.392949 16398 raft_consensus.cc:2258] T 5e84c1913f544a479fdffb4059f7fd54 P 10021826a9bb457b931262ed655abf8c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 16:24:18.394277 16398 ts_tablet_manager.cc:1905] T 5e84c1913f544a479fdffb4059f7fd54 P 10021826a9bb457b931262ed655abf8c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 16:24:18.395926 16398 ts_tablet_manager.cc:1918] T 5e84c1913f544a479fdffb4059f7fd54 P 10021826a9bb457b931262ed655abf8c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 16:24:18.396055 16398 log.cc:1196] T 5e84c1913f544a479fdffb4059f7fd54 P 10021826a9bb457b931262ed655abf8c: Deleting WAL directory at /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786292417722232-21710-0/minicluster-data/ts-2/wal/wals/5e84c1913f544a479fdffb4059f7fd54
I20260809 16:24:18.396363 16398 ts_tablet_manager.cc:1939] T 5e84c1913f544a479fdffb4059f7fd54 P 10021826a9bb457b931262ed655abf8c: Deleting consensus metadata
I20260809 16:24:18.396848 15635 catalog_manager.cc:4841] TS 10021826a9bb457b931262ed655abf8c (127.21.51.131:34159): tablet 5e84c1913f544a479fdffb4059f7fd54 (table test-table [id=732fbcec0f6548498596d54537ab616d]) successfully deleted
W20260809 16:24:18.408253 15639 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.21.51.129:43933 (error 108)
I20260809 16:24:18.408813 16363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:18.408665 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.129:43933 (local address 127.0.0.1:42814)
0809 16:24:18.408735 (+ 70us) negotiation.cc:107] Waiting for socket to connect
0809 16:24:18.408773 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.51.129:43933: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260809 16:24:18.408932 21710 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrNqUCn/build/release/bin/kudu with pid 15959
W20260809 16:24:18.409045 15639 catalog_manager.cc:4584] TS 0c28fcd823f4440aa2a17070dcdb3f19 (127.21.51.129:43933): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 5e84c1913f544a479fdffb4059f7fd54: Network error: Client connection negotiation failed: client connection to 127.21.51.129:43933: connect: Connection refused (error 111)
I20260809 16:24:18.416421 21710 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrNqUCn/build/release/bin/kudu with pid 16159
I20260809 16:24:18.427877 21710 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrNqUCn/build/release/bin/kudu with pid 15612
2026-08-09T16:24:18Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (2310 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-08-09T16:24:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T16:24:18Z Disabled control of system clock
I20260809 16:24:18.461275 21710 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrNqUCn/build/release/bin/kudu
/tmp/dist-test-taskrNqUCn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786292417722232-21710-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786292417722232-21710-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786292417722232-21710-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786292417722232-21710-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.51.190:40055
--webserver_interface=127.21.51.190
--webserver_port=0
--builtin_ntp_servers=127.21.51.148:43443
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.51.190:40055
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 16:24:18.542717 16413 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:24:18.542932 16413 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:24:18.542953 16413 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:24:18.544036 16413 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 16:24:18.544070 16413 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:24:18.544082 16413 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:24:18.544095 16413 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 16:24:18.544108 16413 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 16:24:18.545395 16413 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.51.148:43443
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786292417722232-21710-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786292417722232-21710-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.51.190:40055
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.51.190:40055
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786292417722232-21710-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.51.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16413
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786292417722232-21710-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:24:18.545586 16413 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:24:18.545816 16413 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:24:18.550690 16418 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:24:18.550729 16422 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:24:18.550722 16420 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:24:18.550967 16413 server_base.cc:1034] running on GCE node
I20260809 16:24:18.551227 16413 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:24:18.551548 16413 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:24:18.552739 16413 hybrid_clock.cc:648] HybridClock initialized: now 1786292658552716 us; error 38 us; skew 500 ppm
I20260809 16:24:18.553984 16413 webserver.cc:460] Webserver started at http://127.21.51.190:36819/ using document root <none> and password file <none>
I20260809 16:24:18.554239 16413 fs_manager.cc:359] Metadata directory not provided
I20260809 16:24:18.554309 16413 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:24:18.554463 16413 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:24:18.555581 16413 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786292417722232-21710-0/minicluster-data/master-0/data/instance:
uuid: "14466d615c9a4a47b78a094f7e0db8b3"
format_stamp: "Formatted at 2026-08-09 16:24:18 on dist-test-slave-s11t"
server_key: "ff7264e011e371a17286c4cee805a921"
server_key_iv: "31fb9a330e739f5b918e28f9a3eb8a28"
server_key_version: "encryptionkey@0"
I20260809 16:24:18.555943 16413 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786292417722232-21710-0/minicluster-data/master-0/wal/instance:
uuid: "14466d615c9a4a47b78a094f7e0db8b3"
format_stamp: "Formatted at 2026-08-09 16:24:18 on dist-test-slave-s11t"
server_key: "ff7264e011e371a17286c4cee805a921"
server_key_iv: "31fb9a330e739f5b918e28f9a3eb8a28"
server_key_version: "encryptionkey@0"
I20260809 16:24:18.557312 16413 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 16:24:18.558028 16435 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:24:18.558208 16413 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 16:24:18.558305 16413 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786292417722232-21710-0/minicluster-data/master-0/data,/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786292417722232-21710-0/minicluster-data/master-0/wal
uuid: "14466d615c9a4a47b78a094f7e0db8b3"
format_stamp: "Formatted at 2026-08-09 16:24:18 on dist-test-slave-s11t"
server_key: "ff7264e011e371a17286c4cee805a921"
server_key_iv: "31fb9a330e739f5b918e28f9a3eb8a28"
server_key_version: "encryptionkey@0"
I20260809 16:24:18.558388 16413 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786292417722232-21710-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786292417722232-21710-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786292417722232-21710-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:24:18.573984 16413 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:24:18.574283 16413 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:24:18.574437 16413 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:24:18.578572 16413 rpc_server.cc:307] RPC server started. Bound to: 127.21.51.190:40055
I20260809 16:24:18.578622 16510 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.51.190:40055 every 8 connection(s)
I20260809 16:24:18.578989 16413 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786292417722232-21710-0/minicluster-data/master-0/data/info.pb
I20260809 16:24:18.579998 16511 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:24:18.582410 16511 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 14466d615c9a4a47b78a094f7e0db8b3: Bootstrap starting.
I20260809 16:24:18.583019 16511 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 14466d615c9a4a47b78a094f7e0db8b3: Neither blocks nor log segments found. Creating new log.
I20260809 16:24:18.583328 16511 log.cc:824] T 00000000000000000000000000000000 P 14466d615c9a4a47b78a094f7e0db8b3: Log is configured to *not* fsync() on all Append() calls
I20260809 16:24:18.584028 16511 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 14466d615c9a4a47b78a094f7e0db8b3: No bootstrap required, opened a new log
I20260809 16:24:18.585605 21710 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrNqUCn/build/release/bin/kudu as pid 16413
I20260809 16:24:18.585731 21710 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786292417722232-21710-0/minicluster-data/master-0/wal/instance
I20260809 16:24:18.585675 16511 raft_consensus.cc:348] T 00000000000000000000000000000000 P 14466d615c9a4a47b78a094f7e0db8b3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "14466d615c9a4a47b78a094f7e0db8b3" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 40055 } }
I20260809 16:24:18.585891 16511 raft_consensus.cc:374] T 00000000000000000000000000000000 P 14466d615c9a4a47b78a094f7e0db8b3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:24:18.585920 16511 raft_consensus.cc:729] T 00000000000000000000000000000000 P 14466d615c9a4a47b78a094f7e0db8b3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 14466d615c9a4a47b78a094f7e0db8b3, State: Initialized, Role: FOLLOWER
I20260809 16:24:18.585959 21710 external_mini_cluster.cc:1428] Setting key d5584eca3bc95b8b58aceee4c22f830b
I20260809 16:24:18.586083 16511 consensus_queue.cc:260] T 00000000000000000000000000000000 P 14466d615c9a4a47b78a094f7e0db8b3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "14466d615c9a4a47b78a094f7e0db8b3" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 40055 } }
I20260809 16:24:18.586182 16511 raft_consensus.cc:388] T 00000000000000000000000000000000 P 14466d615c9a4a47b78a094f7e0db8b3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 16:24:18.586220 16511 raft_consensus.cc:482] T 00000000000000000000000000000000 P 14466d615c9a4a47b78a094f7e0db8b3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 16:24:18.586272 16511 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 14466d615c9a4a47b78a094f7e0db8b3 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:24:18.586969 16511 raft_consensus.cc:504] T 00000000000000000000000000000000 P 14466d615c9a4a47b78a094f7e0db8b3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "14466d615c9a4a47b78a094f7e0db8b3" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 40055 } }
I20260809 16:24:18.587083 16511 leader_election.cc:302] T 00000000000000000000000000000000 P 14466d615c9a4a47b78a094f7e0db8b3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 14466d615c9a4a47b78a094f7e0db8b3; no voters:
I20260809 16:24:18.587285 16511 leader_election.cc:288] T 00000000000000000000000000000000 P 14466d615c9a4a47b78a094f7e0db8b3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 16:24:18.587316 16521 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 14466d615c9a4a47b78a094f7e0db8b3 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:24:18.587436 16521 raft_consensus.cc:686] T 00000000000000000000000000000000 P 14466d615c9a4a47b78a094f7e0db8b3 [term 1 LEADER]: Becoming Leader. State: Replica: 14466d615c9a4a47b78a094f7e0db8b3, State: Running, Role: LEADER
I20260809 16:24:18.587580 16511 sys_catalog.cc:564] T 00000000000000000000000000000000 P 14466d615c9a4a47b78a094f7e0db8b3 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 16:24:18.587575 16521 consensus_queue.cc:237] T 00000000000000000000000000000000 P 14466d615c9a4a47b78a094f7e0db8b3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "14466d615c9a4a47b78a094f7e0db8b3" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 40055 } }
I20260809 16:24:18.588089 16523 sys_catalog.cc:455] T 00000000000000000000000000000000 P 14466d615c9a4a47b78a094f7e0db8b3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 14466d615c9a4a47b78a094f7e0db8b3. Latest consensus state: current_term: 1 leader_uuid: "14466d615c9a4a47b78a094f7e0db8b3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "14466d615c9a4a47b78a094f7e0db8b3" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 40055 } } }
I20260809 16:24:18.588184 16523 sys_catalog.cc:458] T 00000000000000000000000000000000 P 14466d615c9a4a47b78a094f7e0db8b3 [sys.catalog]: This master's current role is: LEADER
I20260809 16:24:18.588639 16522 sys_catalog.cc:455] T 00000000000000000000000000000000 P 14466d615c9a4a47b78a094f7e0db8b3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "14466d615c9a4a47b78a094f7e0db8b3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "14466d615c9a4a47b78a094f7e0db8b3" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 40055 } } }
I20260809 16:24:18.588851 16522 sys_catalog.cc:458] T 00000000000000000000000000000000 P 14466d615c9a4a47b78a094f7e0db8b3 [sys.catalog]: This master's current role is: LEADER
I20260809 16:24:18.588928 16528 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 16:24:18.589494 16528 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 16:24:18.592087 16520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:18.586448 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42214 (local address 127.21.51.190:40055)
0809 16:24:18.586708 (+ 260us) server_negotiation.cc:206] Beginning negotiation
0809 16:24:18.586717 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 16:24:18.586753 (+ 36us) server_negotiation.cc:407] Connection header received
0809 16:24:18.586872 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:24:18.586880 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:24:18.586965 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:24:18.587120 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:24:18.587335 (+ 215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:24:18.589168 (+ 1833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:24:18.590042 (+ 874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:24:18.590532 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:24:18.590674 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:24:18.591398 (+ 724us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:24:18.591401 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:24:18.591401 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 16:24:18.591566 (+ 165us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:24:18.591599 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 16:24:18.591608 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:24:18.591663 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0809 16:24:18.591799 (+ 136us) server_negotiation.cc:299] Negotiation successful
0809 16:24:18.591874 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":165,"thread_start_us":112,"threads_started":1}
I20260809 16:24:18.592129 16528 catalog_manager.cc:1347] Generated new cluster ID: 04b60e7dda8f47e5acc56744919e9888
I20260809 16:24:18.592176 16528 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 16:24:18.597554 16528 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 16:24:18.598057 16528 catalog_manager.cc:1504] Loading token signing keys...
I20260809 16:24:18.602550 16528 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 14466d615c9a4a47b78a094f7e0db8b3: Generated new TSK 0
I20260809 16:24:18.602737 16528 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 16:24:18.604513 21710 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrNqUCn/build/release/bin/kudu
/tmp/dist-test-taskrNqUCn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786292417722232-21710-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786292417722232-21710-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786292417722232-21710-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786292417722232-21710-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.51.129:0
--local_ip_for_outbound_sockets=127.21.51.129
--webserver_interface=127.21.51.129
--webserver_port=0
--tserver_master_addrs=127.21.51.190:40055
--builtin_ntp_servers=127.21.51.148:43443
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 16:24:18.696115 16551 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:24:18.696305 16551 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:24:18.696326 16551 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:24:18.697464 16551 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:24:18.697503 16551 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:24:18.697528 16551 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.51.129
I20260809 16:24:18.698836 16551 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.51.148:43443
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786292417722232-21710-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786292417722232-21710-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.51.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786292417722232-21710-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.51.129
--webserver_port=0
--tserver_master_addrs=127.21.51.190:40055
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16551
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.51.129
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786292417722232-21710-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:24:18.699066 16551 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:24:18.699381 16551 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:24:18.705349 16567 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:24:18.705403 16568 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:24:18.705351 16574 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:24:18.705483 16551 server_base.cc:1034] running on GCE node
I20260809 16:24:18.705762 16551 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:24:18.706035 16551 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:24:18.707196 16551 hybrid_clock.cc:648] HybridClock initialized: now 1786292658707179 us; error 34 us; skew 500 ppm
I20260809 16:24:18.708482 16551 webserver.cc:460] Webserver started at http://127.21.51.129:46053/ using document root <none> and password file <none>
I20260809 16:24:18.708796 16551 fs_manager.cc:359] Metadata directory not provided
I20260809 16:24:18.708869 16551 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:24:18.709022 16551 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:24:18.710085 16551 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786292417722232-21710-0/minicluster-data/ts-0/data/instance:
uuid: "b14ea856810949aab8a8c26fecd268d8"
format_stamp: "Formatted at 2026-08-09 16:24:18 on dist-test-slave-s11t"
server_key: "17c399e82366592be1b5f0313ed4efe7"
server_key_iv: "68b76da6da2cf059751486c9b3bae475"
server_key_version: "encryptionkey@0"
I20260809 16:24:18.710486 16551 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786292417722232-21710-0/minicluster-data/ts-0/wal/instance:
uuid: "b14ea856810949aab8a8c26fecd268d8"
format_stamp: "Formatted at 2026-08-09 16:24:18 on dist-test-slave-s11t"
server_key: "17c399e82366592be1b5f0313ed4efe7"
server_key_iv: "68b76da6da2cf059751486c9b3bae475"
server_key_version: "encryptionkey@0"
I20260809 16:24:18.711866 16551 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260809 16:24:18.712673 16582 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:24:18.712852 16551 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:24:18.712944 16551 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786292417722232-21710-0/minicluster-data/ts-0/data,/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786292417722232-21710-0/minicluster-data/ts-0/wal
uuid: "b14ea856810949aab8a8c26fecd268d8"
format_stamp: "Formatted at 2026-08-09 16:24:18 on dist-test-slave-s11t"
server_key: "17c399e82366592be1b5f0313ed4efe7"
server_key_iv: "68b76da6da2cf059751486c9b3bae475"
server_key_version: "encryptionkey@0"
I20260809 16:24:18.713030 16551 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786292417722232-21710-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786292417722232-21710-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786292417722232-21710-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:24:18.740574 16551 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:24:18.740954 16551 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:24:18.741122 16551 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:24:18.741389 16551 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:24:18.741741 16551 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:24:18.741797 16551 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:24:18.741842 16551 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:24:18.741869 16551 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:24:18.747752 16551 rpc_server.cc:307] RPC server started. Bound to: 127.21.51.129:33847
I20260809 16:24:18.747797 16753 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.51.129:33847 every 8 connection(s)
I20260809 16:24:18.748144 16551 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786292417722232-21710-0/minicluster-data/ts-0/data/info.pb
I20260809 16:24:18.751142 21710 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrNqUCn/build/release/bin/kudu as pid 16551
I20260809 16:24:18.751266 21710 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786292417722232-21710-0/minicluster-data/ts-0/wal/instance
I20260809 16:24:18.751479 21710 external_mini_cluster.cc:1428] Setting key 3de9b3c2094c7301cb9fda1b14fec5cd
I20260809 16:24:18.752543 21710 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrNqUCn/build/release/bin/kudu
/tmp/dist-test-taskrNqUCn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786292417722232-21710-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786292417722232-21710-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786292417722232-21710-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786292417722232-21710-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.51.130:0
--local_ip_for_outbound_sockets=127.21.51.130
--webserver_interface=127.21.51.130
--webserver_port=0
--tserver_master_addrs=127.21.51.190:40055
--builtin_ntp_servers=127.21.51.148:43443
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260809 16:24:18.753607 16520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:18.748930 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.51.129:60171 (local address 127.21.51.190:40055)
0809 16:24:18.749019 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0809 16:24:18.749022 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 16:24:18.749226 (+ 204us) server_negotiation.cc:407] Connection header received
0809 16:24:18.749690 (+ 464us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:24:18.749691 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:24:18.749702 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:24:18.749756 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:24:18.750410 (+ 654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:24:18.750903 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:24:18.751973 (+ 1070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:24:18.752265 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:24:18.752329 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:24:18.753283 (+ 954us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:24:18.753285 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:24:18.753285 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 16:24:18.753295 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:24:18.753311 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 16:24:18.753315 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:24:18.753370 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0809 16:24:18.753466 (+ 96us) server_negotiation.cc:299] Negotiation successful
0809 16:24:18.753498 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260809 16:24:18.753827 16757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:18.748895 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:40055 (local address 127.21.51.129:60171)
0809 16:24:18.749104 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0809 16:24:18.749138 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0809 16:24:18.749610 (+ 472us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:24:18.749828 (+ 218us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:24:18.749831 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:24:18.749840 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:24:18.750339 (+ 499us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:24:18.750343 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:24:18.750995 (+ 652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:24:18.750996 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:24:18.751891 (+ 895us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:24:18.751893 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:24:18.752363 (+ 470us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:24:18.752364 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:24:18.752537 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:24:18.753103 (+ 566us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 16:24:18.753104 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:24:18.753144 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 16:24:18.753174 (+ 30us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:24:18.753385 (+ 211us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:24:18.753391 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:24:18.753394 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 16:24:18.753466 (+ 72us) client_negotiation.cc:240] Negotiation successful
0809 16:24:18.753574 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":110,"threads_started":1}
I20260809 16:24:18.754293 16754 heartbeater.cc:347] Connected to a master server at 127.21.51.190:40055
I20260809 16:24:18.754388 16754 heartbeater.cc:464] Registering TS with master...
I20260809 16:24:18.754587 16754 heartbeater.cc:511] Master 127.21.51.190:40055 requested a full tablet report, sending...
I20260809 16:24:18.755060 16458 ts_manager.cc:194] Registered new tserver with Master: b14ea856810949aab8a8c26fecd268d8 (127.21.51.129:33847)
I20260809 16:24:18.755666 16458 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.21.51.129:60171
W20260809 16:24:18.834445 16760 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:24:18.834677 16760 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:24:18.834713 16760 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:24:18.835834 16760 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:24:18.835898 16760 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:24:18.835938 16760 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.51.130
I20260809 16:24:18.837360 16760 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.51.148:43443
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786292417722232-21710-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786292417722232-21710-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.51.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786292417722232-21710-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.51.130
--webserver_port=0
--tserver_master_addrs=127.21.51.190:40055
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16760
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.51.130
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786292417722232-21710-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:24:18.837620 16760 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:24:18.837919 16760 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:24:18.843467 16771 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:24:18.843509 16760 server_base.cc:1034] running on GCE node
W20260809 16:24:18.843474 16770 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:24:18.843636 16773 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:24:18.843935 16760 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:24:18.844230 16760 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:24:18.845395 16760 hybrid_clock.cc:648] HybridClock initialized: now 1786292658845380 us; error 45 us; skew 500 ppm
I20260809 16:24:18.846773 16760 webserver.cc:460] Webserver started at http://127.21.51.130:35757/ using document root <none> and password file <none>
I20260809 16:24:18.847023 16760 fs_manager.cc:359] Metadata directory not provided
I20260809 16:24:18.847098 16760 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:24:18.847260 16760 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:24:18.848330 16760 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786292417722232-21710-0/minicluster-data/ts-1/data/instance:
uuid: "9057b77753d449138fa49b324f9b6929"
format_stamp: "Formatted at 2026-08-09 16:24:18 on dist-test-slave-s11t"
server_key: "0df84911999e2ef6bd5c6da4c4bab85d"
server_key_iv: "d300a6bed5ad5a5517248bd91b5ef0b0"
server_key_version: "encryptionkey@0"
I20260809 16:24:18.848757 16760 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786292417722232-21710-0/minicluster-data/ts-1/wal/instance:
uuid: "9057b77753d449138fa49b324f9b6929"
format_stamp: "Formatted at 2026-08-09 16:24:18 on dist-test-slave-s11t"
server_key: "0df84911999e2ef6bd5c6da4c4bab85d"
server_key_iv: "d300a6bed5ad5a5517248bd91b5ef0b0"
server_key_version: "encryptionkey@0"
I20260809 16:24:18.850052 16760 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260809 16:24:18.850756 16783 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:24:18.850935 16760 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 16:24:18.851034 16760 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786292417722232-21710-0/minicluster-data/ts-1/data,/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786292417722232-21710-0/minicluster-data/ts-1/wal
uuid: "9057b77753d449138fa49b324f9b6929"
format_stamp: "Formatted at 2026-08-09 16:24:18 on dist-test-slave-s11t"
server_key: "0df84911999e2ef6bd5c6da4c4bab85d"
server_key_iv: "d300a6bed5ad5a5517248bd91b5ef0b0"
server_key_version: "encryptionkey@0"
I20260809 16:24:18.851119 16760 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786292417722232-21710-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786292417722232-21710-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786292417722232-21710-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:24:18.862406 16760 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:24:18.862728 16760 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:24:18.862898 16760 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:24:18.863153 16760 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:24:18.863498 16760 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:24:18.863557 16760 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:24:18.863593 16760 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:24:18.863627 16760 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:24:18.869925 16760 rpc_server.cc:307] RPC server started. Bound to: 127.21.51.130:45997
I20260809 16:24:18.869972 16949 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.51.130:45997 every 8 connection(s)
I20260809 16:24:18.870316 16760 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786292417722232-21710-0/minicluster-data/ts-1/data/info.pb
I20260809 16:24:18.875491 16520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:18.871077 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.51.130:40317 (local address 127.21.51.190:40055)
0809 16:24:18.871168 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0809 16:24:18.871171 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 16:24:18.871374 (+ 203us) server_negotiation.cc:407] Connection header received
0809 16:24:18.871833 (+ 459us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:24:18.871834 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:24:18.871846 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:24:18.871905 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:24:18.872615 (+ 710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:24:18.873263 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:24:18.874231 (+ 968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:24:18.874436 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:24:18.874502 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:24:18.875252 (+ 750us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:24:18.875253 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:24:18.875253 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 16:24:18.875258 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:24:18.875271 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 16:24:18.875275 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:24:18.875323 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0809 16:24:18.875408 (+ 85us) server_negotiation.cc:299] Negotiation successful
0809 16:24:18.875431 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260809 16:24:18.875653 16955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:18.871042 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:40055 (local address 127.21.51.130:40317)
0809 16:24:18.871257 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0809 16:24:18.871290 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0809 16:24:18.871741 (+ 451us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:24:18.871976 (+ 235us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:24:18.871980 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:24:18.871988 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:24:18.872521 (+ 533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:24:18.872525 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:24:18.873354 (+ 829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:24:18.873355 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:24:18.874132 (+ 777us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:24:18.874134 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:24:18.874517 (+ 383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:24:18.874518 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:24:18.874626 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:24:18.875120 (+ 494us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 16:24:18.875120 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:24:18.875147 (+ 27us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 16:24:18.875169 (+ 22us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:24:18.875334 (+ 165us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:24:18.875338 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:24:18.875340 (+ 2us) client_negotiation.cc:769] Sending connection context
0809 16:24:18.875390 (+ 50us) client_negotiation.cc:240] Negotiation successful
0809 16:24:18.875470 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":117,"threads_started":1}
I20260809 16:24:18.876169 16950 heartbeater.cc:347] Connected to a master server at 127.21.51.190:40055
I20260809 16:24:18.876271 16950 heartbeater.cc:464] Registering TS with master...
I20260809 16:24:18.876459 16950 heartbeater.cc:511] Master 127.21.51.190:40055 requested a full tablet report, sending...
I20260809 16:24:18.876868 16458 ts_manager.cc:194] Registered new tserver with Master: 9057b77753d449138fa49b324f9b6929 (127.21.51.130:45997)
I20260809 16:24:18.877247 16458 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.21.51.130:40317
I20260809 16:24:18.878355 21710 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrNqUCn/build/release/bin/kudu as pid 16760
I20260809 16:24:18.878484 21710 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786292417722232-21710-0/minicluster-data/ts-1/wal/instance
I20260809 16:24:18.878700 21710 external_mini_cluster.cc:1428] Setting key 27d2633bb3b404dc9776478eee909277
I20260809 16:24:18.879534 21710 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrNqUCn/build/release/bin/kudu
/tmp/dist-test-taskrNqUCn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786292417722232-21710-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786292417722232-21710-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786292417722232-21710-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786292417722232-21710-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.51.131:0
--local_ip_for_outbound_sockets=127.21.51.131
--webserver_interface=127.21.51.131
--webserver_port=0
--tserver_master_addrs=127.21.51.190:40055
--builtin_ntp_servers=127.21.51.148:43443
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 16:24:18.962364 16956 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:24:18.962556 16956 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:24:18.962579 16956 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:24:18.963698 16956 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:24:18.963738 16956 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:24:18.963763 16956 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.51.131
I20260809 16:24:18.965152 16956 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.51.148:43443
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786292417722232-21710-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786292417722232-21710-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.51.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786292417722232-21710-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.51.131
--webserver_port=0
--tserver_master_addrs=127.21.51.190:40055
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16956
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.51.131
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786292417722232-21710-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:24:18.965379 16956 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:24:18.965688 16956 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:24:18.971201 16966 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:24:18.971172 16964 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:24:18.971184 16968 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:24:18.971383 16956 server_base.cc:1034] running on GCE node
I20260809 16:24:18.971604 16956 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:24:18.971844 16956 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:24:18.972999 16956 hybrid_clock.cc:648] HybridClock initialized: now 1786292658972989 us; error 46 us; skew 500 ppm
I20260809 16:24:18.974296 16956 webserver.cc:460] Webserver started at http://127.21.51.131:37437/ using document root <none> and password file <none>
I20260809 16:24:18.974586 16956 fs_manager.cc:359] Metadata directory not provided
I20260809 16:24:18.974658 16956 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:24:18.974813 16956 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:24:18.975991 16956 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786292417722232-21710-0/minicluster-data/ts-2/data/instance:
uuid: "0f0d3a27019f41c6bad1daf784349767"
format_stamp: "Formatted at 2026-08-09 16:24:18 on dist-test-slave-s11t"
server_key: "57016f53ceb6dbfb3ba799ed8a4c39c4"
server_key_iv: "3a366170129d65abb5aad447b5d0fbb3"
server_key_version: "encryptionkey@0"
I20260809 16:24:18.976357 16956 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786292417722232-21710-0/minicluster-data/ts-2/wal/instance:
uuid: "0f0d3a27019f41c6bad1daf784349767"
format_stamp: "Formatted at 2026-08-09 16:24:18 on dist-test-slave-s11t"
server_key: "57016f53ceb6dbfb3ba799ed8a4c39c4"
server_key_iv: "3a366170129d65abb5aad447b5d0fbb3"
server_key_version: "encryptionkey@0"
I20260809 16:24:18.977770 16956 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 16:24:18.978546 16977 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:24:18.978735 16956 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 16:24:18.978842 16956 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786292417722232-21710-0/minicluster-data/ts-2/data,/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786292417722232-21710-0/minicluster-data/ts-2/wal
uuid: "0f0d3a27019f41c6bad1daf784349767"
format_stamp: "Formatted at 2026-08-09 16:24:18 on dist-test-slave-s11t"
server_key: "57016f53ceb6dbfb3ba799ed8a4c39c4"
server_key_iv: "3a366170129d65abb5aad447b5d0fbb3"
server_key_version: "encryptionkey@0"
I20260809 16:24:18.978915 16956 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786292417722232-21710-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786292417722232-21710-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786292417722232-21710-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:24:19.002982 16956 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:24:19.003348 16956 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:24:19.003515 16956 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:24:19.003774 16956 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:24:19.004129 16956 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:24:19.004186 16956 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:24:19.004230 16956 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:24:19.004257 16956 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:24:19.010141 16956 rpc_server.cc:307] RPC server started. Bound to: 127.21.51.131:45309
I20260809 16:24:19.010188 17146 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.51.131:45309 every 8 connection(s)
I20260809 16:24:19.010536 16956 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786292417722232-21710-0/minicluster-data/ts-2/data/info.pb
I20260809 16:24:19.015139 21710 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrNqUCn/build/release/bin/kudu as pid 16956
I20260809 16:24:19.015275 21710 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786292417722232-21710-0/minicluster-data/ts-2/wal/instance
I20260809 16:24:19.015529 21710 external_mini_cluster.cc:1428] Setting key 7d2b4579e49cf1d1118db3c7a06613ee
I20260809 16:24:19.015887 16520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:19.011253 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.51.131:38979 (local address 127.21.51.190:40055)
0809 16:24:19.011342 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0809 16:24:19.011344 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:24:19.011552 (+ 208us) server_negotiation.cc:407] Connection header received
0809 16:24:19.012047 (+ 495us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:24:19.012048 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:24:19.012059 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:24:19.012112 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:24:19.012839 (+ 727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:24:19.013343 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:24:19.014325 (+ 982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:24:19.014528 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:24:19.014594 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:24:19.015510 (+ 916us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:24:19.015511 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:24:19.015511 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 16:24:19.015515 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:24:19.015526 (+ 11us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 16:24:19.015529 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:24:19.015574 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0809 16:24:19.015755 (+ 181us) server_negotiation.cc:299] Negotiation successful
0809 16:24:19.015787 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260809 16:24:19.016047 17151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:19.011215 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:40055 (local address 127.21.51.131:38979)
0809 16:24:19.011428 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0809 16:24:19.011464 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0809 16:24:19.011958 (+ 494us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:24:19.012172 (+ 214us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:24:19.012175 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:24:19.012183 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:24:19.012750 (+ 567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:24:19.012756 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:24:19.013423 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:24:19.013424 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:24:19.014228 (+ 804us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:24:19.014230 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:24:19.014612 (+ 382us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:24:19.014613 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:24:19.014722 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:24:19.015314 (+ 592us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 16:24:19.015315 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:24:19.015371 (+ 56us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 16:24:19.015404 (+ 33us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:24:19.015598 (+ 194us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:24:19.015605 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:24:19.015609 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 16:24:19.015700 (+ 91us) client_negotiation.cc:240] Negotiation successful
0809 16:24:19.015808 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":111,"threads_started":1}
I20260809 16:24:19.016512 17147 heartbeater.cc:347] Connected to a master server at 127.21.51.190:40055
I20260809 16:24:19.016609 17147 heartbeater.cc:464] Registering TS with master...
I20260809 16:24:19.016813 17147 heartbeater.cc:511] Master 127.21.51.190:40055 requested a full tablet report, sending...
I20260809 16:24:19.017230 16457 ts_manager.cc:194] Registered new tserver with Master: 0f0d3a27019f41c6bad1daf784349767 (127.21.51.131:45309)
I20260809 16:24:19.017567 16457 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.21.51.131:38979
I20260809 16:24:19.027491 21710 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 16:24:19.031973 16520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:19.029185 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42220 (local address 127.21.51.190:40055)
0809 16:24:19.029278 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0809 16:24:19.029280 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:24:19.029346 (+ 66us) server_negotiation.cc:407] Connection header received
0809 16:24:19.029449 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:24:19.029450 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:24:19.029479 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:24:19.029540 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:24:19.029728 (+ 188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:24:19.030248 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:24:19.030988 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:24:19.031163 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:24:19.031217 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:24:19.031771 (+ 554us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:24:19.031772 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:24:19.031772 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 16:24:19.031777 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:24:19.031789 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 16:24:19.031792 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:24:19.031858 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0809 16:24:19.031896 (+ 38us) server_negotiation.cc:299] Negotiation successful
0809 16:24:19.031916 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260809 16:24:19.033718 16457 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:42220:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 16:24:19.035238 16457 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 16:24:19.041842 17159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:19.037468 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.130:45997 (local address 127.0.0.1:38298)
0809 16:24:19.037645 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0809 16:24:19.037687 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0809 16:24:19.037753 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:24:19.038272 (+ 519us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:24:19.038275 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:24:19.038277 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:24:19.038340 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:24:19.038342 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:24:19.039271 (+ 929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:24:19.039273 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:24:19.040554 (+ 1281us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:24:19.040557 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:24:19.041536 (+ 979us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:24:19.041537 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:24:19.041624 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:24:19.041639 (+ 15us) client_negotiation.cc:769] Sending connection context
0809 16:24:19.041717 (+ 78us) client_negotiation.cc:240] Negotiation successful
0809 16:24:19.041769 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":152,"mutex_wait_us":5,"thread_start_us":85,"threads_started":1}
I20260809 16:24:19.041986 17160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:19.037512 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.129:33847 (local address 127.0.0.1:53006)
0809 16:24:19.037867 (+ 355us) negotiation.cc:107] Waiting for socket to connect
0809 16:24:19.037877 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 16:24:19.037961 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:24:19.038294 (+ 333us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:24:19.038295 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:24:19.038296 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:24:19.038344 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:24:19.038345 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:24:19.039672 (+ 1327us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:24:19.039673 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:24:19.040776 (+ 1103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:24:19.040782 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:24:19.041729 (+ 947us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:24:19.041730 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:24:19.041790 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:24:19.041793 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 16:24:19.041845 (+ 52us) client_negotiation.cc:240] Negotiation successful
0809 16:24:19.041886 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":310,"thread_start_us":88,"threads_started":1}
I20260809 16:24:19.042217 17163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:19.037672 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53006 (local address 127.21.51.129:33847)
0809 16:24:19.037810 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0809 16:24:19.037815 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 16:24:19.038077 (+ 262us) server_negotiation.cc:407] Connection header received
0809 16:24:19.038128 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:24:19.038130 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:24:19.038146 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:24:19.038224 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:24:19.038428 (+ 204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:24:19.039586 (+ 1158us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:24:19.040849 (+ 1263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:24:19.041556 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:24:19.041616 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:24:19.041690 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0809 16:24:19.042091 (+ 401us) server_negotiation.cc:299] Negotiation successful
0809 16:24:19.042138 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":101,"thread_start_us":69,"threads_started":1}
I20260809 16:24:19.042394 17162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:19.037564 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38298 (local address 127.21.51.130:45997)
0809 16:24:19.038097 (+ 533us) server_negotiation.cc:206] Beginning negotiation
0809 16:24:19.038101 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 16:24:19.038110 (+ 9us) server_negotiation.cc:407] Connection header received
0809 16:24:19.038147 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:24:19.038149 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:24:19.038163 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:24:19.038425 (+ 262us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:24:19.038491 (+ 66us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:24:19.039164 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:24:19.040643 (+ 1479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:24:19.041439 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:24:19.041523 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:24:19.041622 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0809 16:24:19.041734 (+ 112us) server_negotiation.cc:299] Negotiation successful
0809 16:24:19.041773 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":485,"thread_start_us":86,"threads_started":1}
I20260809 16:24:19.042938 16661 tablet_service.cc:1467] Processing CreateTablet for tablet 8d6b7976043647df81bc2b8346618b2d (DEFAULT_TABLE table=test-table [id=423cb63b64bf4835bd9aebd58c0512ed]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:24:19.043015 16857 tablet_service.cc:1467] Processing CreateTablet for tablet 8d6b7976043647df81bc2b8346618b2d (DEFAULT_TABLE table=test-table [id=423cb63b64bf4835bd9aebd58c0512ed]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:24:19.043347 16661 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8d6b7976043647df81bc2b8346618b2d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:24:19.043354 16857 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8d6b7976043647df81bc2b8346618b2d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:24:19.045195 17164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:19.037693 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.131:45309 (local address 127.0.0.1:39148)
0809 16:24:19.037933 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0809 16:24:19.037944 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 16:24:19.037993 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:24:19.038311 (+ 318us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:24:19.038312 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:24:19.038314 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:24:19.038361 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:24:19.038363 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:24:19.040855 (+ 2492us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:24:19.040855 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:24:19.041535 (+ 680us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:24:19.041536 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:24:19.044943 (+ 3407us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:24:19.044944 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:24:19.045006 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:24:19.045010 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 16:24:19.045067 (+ 57us) client_negotiation.cc:240] Negotiation successful
0809 16:24:19.045115 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":222,"thread_start_us":88,"threads_started":1}
I20260809 16:24:19.045527 17165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:19.037899 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39148 (local address 127.21.51.131:45309)
0809 16:24:19.038103 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0809 16:24:19.038108 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 16:24:19.038117 (+ 9us) server_negotiation.cc:407] Connection header received
0809 16:24:19.038153 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:24:19.038155 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:24:19.038171 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:24:19.038247 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:24:19.038471 (+ 224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:24:19.039577 (+ 1106us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:24:19.041627 (+ 2050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:24:19.044844 (+ 3217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:24:19.044934 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:24:19.045020 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0809 16:24:19.045350 (+ 330us) server_negotiation.cc:299] Negotiation successful
0809 16:24:19.045414 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":171,"thread_start_us":98,"threads_started":1}
I20260809 16:24:19.046120 17043 tablet_service.cc:1467] Processing CreateTablet for tablet 8d6b7976043647df81bc2b8346618b2d (DEFAULT_TABLE table=test-table [id=423cb63b64bf4835bd9aebd58c0512ed]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:24:19.046439 17043 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8d6b7976043647df81bc2b8346618b2d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:24:19.046787 17166 tablet_bootstrap.cc:492] T 8d6b7976043647df81bc2b8346618b2d P 9057b77753d449138fa49b324f9b6929: Bootstrap starting.
I20260809 16:24:19.047008 17168 tablet_bootstrap.cc:492] T 8d6b7976043647df81bc2b8346618b2d P b14ea856810949aab8a8c26fecd268d8: Bootstrap starting.
I20260809 16:24:19.047717 17168 tablet_bootstrap.cc:654] T 8d6b7976043647df81bc2b8346618b2d P b14ea856810949aab8a8c26fecd268d8: Neither blocks nor log segments found. Creating new log.
I20260809 16:24:19.048065 17168 log.cc:824] T 8d6b7976043647df81bc2b8346618b2d P b14ea856810949aab8a8c26fecd268d8: Log is configured to *not* fsync() on all Append() calls
I20260809 16:24:19.048954 17168 tablet_bootstrap.cc:492] T 8d6b7976043647df81bc2b8346618b2d P b14ea856810949aab8a8c26fecd268d8: No bootstrap required, opened a new log
I20260809 16:24:19.049002 17166 tablet_bootstrap.cc:654] T 8d6b7976043647df81bc2b8346618b2d P 9057b77753d449138fa49b324f9b6929: Neither blocks nor log segments found. Creating new log.
I20260809 16:24:19.049044 17168 ts_tablet_manager.cc:1397] T 8d6b7976043647df81bc2b8346618b2d P b14ea856810949aab8a8c26fecd268d8: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 16:24:19.049304 17166 log.cc:824] T 8d6b7976043647df81bc2b8346618b2d P 9057b77753d449138fa49b324f9b6929: Log is configured to *not* fsync() on all Append() calls
I20260809 16:24:19.050069 17172 tablet_bootstrap.cc:492] T 8d6b7976043647df81bc2b8346618b2d P 0f0d3a27019f41c6bad1daf784349767: Bootstrap starting.
I20260809 16:24:19.050173 17166 tablet_bootstrap.cc:492] T 8d6b7976043647df81bc2b8346618b2d P 9057b77753d449138fa49b324f9b6929: No bootstrap required, opened a new log
I20260809 16:24:19.050247 17166 ts_tablet_manager.cc:1397] T 8d6b7976043647df81bc2b8346618b2d P 9057b77753d449138fa49b324f9b6929: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.001s
I20260809 16:24:19.050760 17172 tablet_bootstrap.cc:654] T 8d6b7976043647df81bc2b8346618b2d P 0f0d3a27019f41c6bad1daf784349767: Neither blocks nor log segments found. Creating new log.
I20260809 16:24:19.051070 17172 log.cc:824] T 8d6b7976043647df81bc2b8346618b2d P 0f0d3a27019f41c6bad1daf784349767: Log is configured to *not* fsync() on all Append() calls
I20260809 16:24:19.051846 17172 tablet_bootstrap.cc:492] T 8d6b7976043647df81bc2b8346618b2d P 0f0d3a27019f41c6bad1daf784349767: No bootstrap required, opened a new log
I20260809 16:24:19.051766 17168 raft_consensus.cc:348] T 8d6b7976043647df81bc2b8346618b2d P b14ea856810949aab8a8c26fecd268d8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9057b77753d449138fa49b324f9b6929" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 45997 } } peers { permanent_uuid: "0f0d3a27019f41c6bad1daf784349767" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 45309 } } peers { permanent_uuid: "b14ea856810949aab8a8c26fecd268d8" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 33847 } }
I20260809 16:24:19.051911 17172 ts_tablet_manager.cc:1397] T 8d6b7976043647df81bc2b8346618b2d P 0f0d3a27019f41c6bad1daf784349767: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 16:24:19.051936 17168 raft_consensus.cc:374] T 8d6b7976043647df81bc2b8346618b2d P b14ea856810949aab8a8c26fecd268d8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:24:19.051988 17168 raft_consensus.cc:729] T 8d6b7976043647df81bc2b8346618b2d P b14ea856810949aab8a8c26fecd268d8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b14ea856810949aab8a8c26fecd268d8, State: Initialized, Role: FOLLOWER
I20260809 16:24:19.052117 17168 consensus_queue.cc:260] T 8d6b7976043647df81bc2b8346618b2d P b14ea856810949aab8a8c26fecd268d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9057b77753d449138fa49b324f9b6929" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 45997 } } peers { permanent_uuid: "0f0d3a27019f41c6bad1daf784349767" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 45309 } } peers { permanent_uuid: "b14ea856810949aab8a8c26fecd268d8" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 33847 } }
I20260809 16:24:19.052384 17168 ts_tablet_manager.cc:1428] T 8d6b7976043647df81bc2b8346618b2d P b14ea856810949aab8a8c26fecd268d8: Time spent starting tablet: real 0.003s user 0.000s sys 0.004s
I20260809 16:24:19.052520 16754 heartbeater.cc:503] Master 127.21.51.190:40055 was elected leader, sending a full tablet report...
I20260809 16:24:19.052498 17166 raft_consensus.cc:348] T 8d6b7976043647df81bc2b8346618b2d P 9057b77753d449138fa49b324f9b6929 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9057b77753d449138fa49b324f9b6929" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 45997 } } peers { permanent_uuid: "0f0d3a27019f41c6bad1daf784349767" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 45309 } } peers { permanent_uuid: "b14ea856810949aab8a8c26fecd268d8" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 33847 } }
I20260809 16:24:19.052651 17166 raft_consensus.cc:374] T 8d6b7976043647df81bc2b8346618b2d P 9057b77753d449138fa49b324f9b6929 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:24:19.052712 17166 raft_consensus.cc:729] T 8d6b7976043647df81bc2b8346618b2d P 9057b77753d449138fa49b324f9b6929 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9057b77753d449138fa49b324f9b6929, State: Initialized, Role: FOLLOWER
I20260809 16:24:19.052829 17166 consensus_queue.cc:260] T 8d6b7976043647df81bc2b8346618b2d P 9057b77753d449138fa49b324f9b6929 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9057b77753d449138fa49b324f9b6929" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 45997 } } peers { permanent_uuid: "0f0d3a27019f41c6bad1daf784349767" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 45309 } } peers { permanent_uuid: "b14ea856810949aab8a8c26fecd268d8" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 33847 } }
I20260809 16:24:19.053094 17166 ts_tablet_manager.cc:1428] T 8d6b7976043647df81bc2b8346618b2d P 9057b77753d449138fa49b324f9b6929: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260809 16:24:19.053298 16950 heartbeater.cc:503] Master 127.21.51.190:40055 was elected leader, sending a full tablet report...
I20260809 16:24:19.054572 17172 raft_consensus.cc:348] T 8d6b7976043647df81bc2b8346618b2d P 0f0d3a27019f41c6bad1daf784349767 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9057b77753d449138fa49b324f9b6929" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 45997 } } peers { permanent_uuid: "0f0d3a27019f41c6bad1daf784349767" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 45309 } } peers { permanent_uuid: "b14ea856810949aab8a8c26fecd268d8" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 33847 } }
I20260809 16:24:19.054684 17172 raft_consensus.cc:374] T 8d6b7976043647df81bc2b8346618b2d P 0f0d3a27019f41c6bad1daf784349767 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:24:19.054705 17172 raft_consensus.cc:729] T 8d6b7976043647df81bc2b8346618b2d P 0f0d3a27019f41c6bad1daf784349767 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0f0d3a27019f41c6bad1daf784349767, State: Initialized, Role: FOLLOWER
I20260809 16:24:19.054826 17172 consensus_queue.cc:260] T 8d6b7976043647df81bc2b8346618b2d P 0f0d3a27019f41c6bad1daf784349767 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9057b77753d449138fa49b324f9b6929" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 45997 } } peers { permanent_uuid: "0f0d3a27019f41c6bad1daf784349767" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 45309 } } peers { permanent_uuid: "b14ea856810949aab8a8c26fecd268d8" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 33847 } }
I20260809 16:24:19.055109 17172 ts_tablet_manager.cc:1428] T 8d6b7976043647df81bc2b8346618b2d P 0f0d3a27019f41c6bad1daf784349767: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 16:24:19.055173 17147 heartbeater.cc:503] Master 127.21.51.190:40055 was elected leader, sending a full tablet report...
I20260809 16:24:19.099472 17174 raft_consensus.cc:482] T 8d6b7976043647df81bc2b8346618b2d P b14ea856810949aab8a8c26fecd268d8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 16:24:19.099602 17174 raft_consensus.cc:504] T 8d6b7976043647df81bc2b8346618b2d P b14ea856810949aab8a8c26fecd268d8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9057b77753d449138fa49b324f9b6929" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 45997 } } peers { permanent_uuid: "0f0d3a27019f41c6bad1daf784349767" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 45309 } } peers { permanent_uuid: "b14ea856810949aab8a8c26fecd268d8" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 33847 } }
I20260809 16:24:19.099900 17174 leader_election.cc:288] T 8d6b7976043647df81bc2b8346618b2d P b14ea856810949aab8a8c26fecd268d8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9057b77753d449138fa49b324f9b6929 (127.21.51.130:45997), 0f0d3a27019f41c6bad1daf784349767 (127.21.51.131:45309)
I20260809 16:24:19.102824 17162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:19.100206 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.51.129:45287 (local address 127.21.51.130:45997)
0809 16:24:19.100288 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0809 16:24:19.100290 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:24:19.100300 (+ 10us) server_negotiation.cc:407] Connection header received
0809 16:24:19.100331 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:24:19.100331 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:24:19.100344 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:24:19.100405 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:24:19.100583 (+ 178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:24:19.101155 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:24:19.102088 (+ 933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:24:19.102460 (+ 372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:24:19.102550 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:24:19.102559 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0809 16:24:19.102716 (+ 157us) server_negotiation.cc:299] Negotiation successful
0809 16:24:19.102754 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260809 16:24:19.102825 16757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:19.100089 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.130:45997 (local address 127.21.51.129:45287)
0809 16:24:19.100156 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0809 16:24:19.100169 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 16:24:19.100230 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:24:19.100435 (+ 205us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:24:19.100436 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:24:19.100438 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:24:19.100517 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:24:19.100519 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:24:19.101246 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:24:19.101248 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:24:19.101994 (+ 746us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:24:19.101996 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:24:19.102550 (+ 554us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:24:19.102551 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:24:19.102628 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:24:19.102632 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 16:24:19.102711 (+ 79us) client_negotiation.cc:240] Negotiation successful
0809 16:24:19.102752 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43,"mutex_wait_us":1}
I20260809 16:24:19.103252 17179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:19.100198 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.131:45309 (local address 127.21.51.129:59679)
0809 16:24:19.100380 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0809 16:24:19.100407 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0809 16:24:19.100478 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:24:19.100616 (+ 138us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:24:19.100617 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:24:19.100619 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:24:19.100755 (+ 136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:24:19.100757 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:24:19.101522 (+ 765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:24:19.101523 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:24:19.102248 (+ 725us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:24:19.102250 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:24:19.102805 (+ 555us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:24:19.102806 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:24:19.102917 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:24:19.103072 (+ 155us) client_negotiation.cc:769] Sending connection context
0809 16:24:19.103134 (+ 62us) client_negotiation.cc:240] Negotiation successful
0809 16:24:19.103175 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":122,"threads_started":1}
I20260809 16:24:19.103206 16883 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d6b7976043647df81bc2b8346618b2d" candidate_uuid: "b14ea856810949aab8a8c26fecd268d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9057b77753d449138fa49b324f9b6929" is_pre_election: true
I20260809 16:24:19.103271 17165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:19.100282 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.51.129:59679 (local address 127.21.51.131:45309)
0809 16:24:19.100329 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0809 16:24:19.100332 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 16:24:19.100487 (+ 155us) server_negotiation.cc:407] Connection header received
0809 16:24:19.100537 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:24:19.100537 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:24:19.100545 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:24:19.100610 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:24:19.100876 (+ 266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:24:19.101440 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:24:19.102337 (+ 897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:24:19.102706 (+ 369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:24:19.102790 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:24:19.102798 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0809 16:24:19.103159 (+ 361us) server_negotiation.cc:299] Negotiation successful
0809 16:24:19.103208 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260809 16:24:19.103384 16883 raft_consensus.cc:2454] T 8d6b7976043647df81bc2b8346618b2d P 9057b77753d449138fa49b324f9b6929 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b14ea856810949aab8a8c26fecd268d8 in term 0.
I20260809 16:24:19.103618 16589 leader_election.cc:302] T 8d6b7976043647df81bc2b8346618b2d P b14ea856810949aab8a8c26fecd268d8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9057b77753d449138fa49b324f9b6929, b14ea856810949aab8a8c26fecd268d8; no voters:
I20260809 16:24:19.103591 17076 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d6b7976043647df81bc2b8346618b2d" candidate_uuid: "b14ea856810949aab8a8c26fecd268d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0f0d3a27019f41c6bad1daf784349767" is_pre_election: true
I20260809 16:24:19.103729 17076 raft_consensus.cc:2454] T 8d6b7976043647df81bc2b8346618b2d P 0f0d3a27019f41c6bad1daf784349767 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b14ea856810949aab8a8c26fecd268d8 in term 0.
I20260809 16:24:19.103775 17174 raft_consensus.cc:2781] T 8d6b7976043647df81bc2b8346618b2d P b14ea856810949aab8a8c26fecd268d8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 16:24:19.103832 17174 raft_consensus.cc:482] T 8d6b7976043647df81bc2b8346618b2d P b14ea856810949aab8a8c26fecd268d8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 16:24:19.103865 17174 raft_consensus.cc:3037] T 8d6b7976043647df81bc2b8346618b2d P b14ea856810949aab8a8c26fecd268d8 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:24:19.104588 17174 raft_consensus.cc:504] T 8d6b7976043647df81bc2b8346618b2d P b14ea856810949aab8a8c26fecd268d8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9057b77753d449138fa49b324f9b6929" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 45997 } } peers { permanent_uuid: "0f0d3a27019f41c6bad1daf784349767" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 45309 } } peers { permanent_uuid: "b14ea856810949aab8a8c26fecd268d8" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 33847 } }
I20260809 16:24:19.104748 17174 leader_election.cc:288] T 8d6b7976043647df81bc2b8346618b2d P b14ea856810949aab8a8c26fecd268d8 [CANDIDATE]: Term 1 election: Requested vote from peers 9057b77753d449138fa49b324f9b6929 (127.21.51.130:45997), 0f0d3a27019f41c6bad1daf784349767 (127.21.51.131:45309)
I20260809 16:24:19.104938 17076 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d6b7976043647df81bc2b8346618b2d" candidate_uuid: "b14ea856810949aab8a8c26fecd268d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0f0d3a27019f41c6bad1daf784349767"
I20260809 16:24:19.105015 17076 raft_consensus.cc:3037] T 8d6b7976043647df81bc2b8346618b2d P 0f0d3a27019f41c6bad1daf784349767 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:24:19.105729 17076 raft_consensus.cc:2454] T 8d6b7976043647df81bc2b8346618b2d P 0f0d3a27019f41c6bad1daf784349767 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b14ea856810949aab8a8c26fecd268d8 in term 1.
I20260809 16:24:19.105911 16589 leader_election.cc:302] T 8d6b7976043647df81bc2b8346618b2d P b14ea856810949aab8a8c26fecd268d8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f0d3a27019f41c6bad1daf784349767, b14ea856810949aab8a8c26fecd268d8; no voters:
I20260809 16:24:19.105878 16883 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d6b7976043647df81bc2b8346618b2d" candidate_uuid: "b14ea856810949aab8a8c26fecd268d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9057b77753d449138fa49b324f9b6929"
I20260809 16:24:19.105981 16883 raft_consensus.cc:3037] T 8d6b7976043647df81bc2b8346618b2d P 9057b77753d449138fa49b324f9b6929 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:24:19.106065 17174 raft_consensus.cc:2781] T 8d6b7976043647df81bc2b8346618b2d P b14ea856810949aab8a8c26fecd268d8 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:24:19.106220 17174 raft_consensus.cc:686] T 8d6b7976043647df81bc2b8346618b2d P b14ea856810949aab8a8c26fecd268d8 [term 1 LEADER]: Becoming Leader. State: Replica: b14ea856810949aab8a8c26fecd268d8, State: Running, Role: LEADER
I20260809 16:24:19.106318 17174 consensus_queue.cc:237] T 8d6b7976043647df81bc2b8346618b2d P b14ea856810949aab8a8c26fecd268d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9057b77753d449138fa49b324f9b6929" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 45997 } } peers { permanent_uuid: "0f0d3a27019f41c6bad1daf784349767" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 45309 } } peers { permanent_uuid: "b14ea856810949aab8a8c26fecd268d8" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 33847 } }
I20260809 16:24:19.106971 16883 raft_consensus.cc:2454] T 8d6b7976043647df81bc2b8346618b2d P 9057b77753d449138fa49b324f9b6929 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b14ea856810949aab8a8c26fecd268d8 in term 1.
I20260809 16:24:19.107256 16457 catalog_manager.cc:5495] T 8d6b7976043647df81bc2b8346618b2d P b14ea856810949aab8a8c26fecd268d8 reported cstate change: term changed from 0 to 1, leader changed from <none> to b14ea856810949aab8a8c26fecd268d8 (127.21.51.129). New cstate: current_term: 1 leader_uuid: "b14ea856810949aab8a8c26fecd268d8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9057b77753d449138fa49b324f9b6929" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 45997 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0f0d3a27019f41c6bad1daf784349767" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 45309 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b14ea856810949aab8a8c26fecd268d8" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 33847 } health_report { overall_health: HEALTHY } } }
I20260809 16:24:19.112632 17163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:19.110607 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53014 (local address 127.21.51.129:33847)
0809 16:24:19.110657 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0809 16:24:19.110659 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:24:19.110668 (+ 9us) server_negotiation.cc:407] Connection header received
0809 16:24:19.110698 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:24:19.110698 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:24:19.110721 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:24:19.110782 (+ 61us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 16:24:19.110909 (+ 127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:24:19.111338 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:24:19.112024 (+ 686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:24:19.112195 (+ 171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:24:19.112242 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:24:19.112369 (+ 127us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 16:24:19.112463 (+ 94us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 16:24:19.112520 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0809 16:24:19.112552 (+ 32us) server_negotiation.cc:299] Negotiation successful
0809 16:24:19.112568 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260809 16:24:19.114292 17076 raft_consensus.cc:1261] T 8d6b7976043647df81bc2b8346618b2d P 0f0d3a27019f41c6bad1daf784349767 [term 1 FOLLOWER]: Refusing update from remote peer b14ea856810949aab8a8c26fecd268d8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 16:24:19.114312 16883 raft_consensus.cc:1261] T 8d6b7976043647df81bc2b8346618b2d P 9057b77753d449138fa49b324f9b6929 [term 1 FOLLOWER]: Refusing update from remote peer b14ea856810949aab8a8c26fecd268d8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 16:24:19.114631 17174 consensus_queue.cc:1035] T 8d6b7976043647df81bc2b8346618b2d P b14ea856810949aab8a8c26fecd268d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9057b77753d449138fa49b324f9b6929" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 45997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 16:24:19.114780 17180 consensus_queue.cc:1035] T 8d6b7976043647df81bc2b8346618b2d P b14ea856810949aab8a8c26fecd268d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f0d3a27019f41c6bad1daf784349767" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 45309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 16:24:19.118367 17187 mvcc.cc:205] Tried to move back new op lower bound from 7316654731729739776 to 7316654731700142080. Current Snapshot: MvccSnapshot[applied={T|T < 7316654731729739776 or (T in {7316654731729739776})}]
I20260809 16:24:19.118494 17185 mvcc.cc:205] Tried to move back new op lower bound from 7316654731729739776 to 7316654731700142080. Current Snapshot: MvccSnapshot[applied={T|T < 7316654731729739776 or (T in {7316654731729739776})}]
I20260809 16:24:19.119596 16458 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:42220:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 16:24:19.119704 16458 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:42220:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 16:24:19.120438 16458 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 14466d615c9a4a47b78a094f7e0db8b3: Sending DeleteTablet for 3 replicas of tablet 8d6b7976043647df81bc2b8346618b2d
I20260809 16:24:19.120817 17043 tablet_service.cc:1514] Processing DeleteTablet for tablet 8d6b7976043647df81bc2b8346618b2d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:24:19 UTC) from {username='slave'} at 127.0.0.1:39148
I20260809 16:24:19.120798 16857 tablet_service.cc:1514] Processing DeleteTablet for tablet 8d6b7976043647df81bc2b8346618b2d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:24:19 UTC) from {username='slave'} at 127.0.0.1:38298
I20260809 16:24:19.121024 17197 tablet_replica.cc:330] T 8d6b7976043647df81bc2b8346618b2d P 9057b77753d449138fa49b324f9b6929: stopping tablet replica
I20260809 16:24:19.121025 16661 tablet_service.cc:1514] Processing DeleteTablet for tablet 8d6b7976043647df81bc2b8346618b2d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:24:19 UTC) from {username='slave'} at 127.0.0.1:53006
I20260809 16:24:19.121064 17198 tablet_replica.cc:330] T 8d6b7976043647df81bc2b8346618b2d P 0f0d3a27019f41c6bad1daf784349767: stopping tablet replica
I20260809 16:24:19.121124 17197 raft_consensus.cc:2229] T 8d6b7976043647df81bc2b8346618b2d P 9057b77753d449138fa49b324f9b6929 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 16:24:19.121213 17198 raft_consensus.cc:2229] T 8d6b7976043647df81bc2b8346618b2d P 0f0d3a27019f41c6bad1daf784349767 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 16:24:19.121232 17197 raft_consensus.cc:2258] T 8d6b7976043647df81bc2b8346618b2d P 9057b77753d449138fa49b324f9b6929 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 16:24:19.121529 17201 tablet_replica.cc:330] T 8d6b7976043647df81bc2b8346618b2d P b14ea856810949aab8a8c26fecd268d8: stopping tablet replica
I20260809 16:24:19.121587 21710 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrNqUCn/build/release/bin/kudu with pid 16551
I20260809 16:24:19.121625 17201 raft_consensus.cc:2229] T 8d6b7976043647df81bc2b8346618b2d P b14ea856810949aab8a8c26fecd268d8 [term 1 LEADER]: Raft consensus shutting down.
I20260809 16:24:19.121709 17201 raft_consensus.cc:2258] T 8d6b7976043647df81bc2b8346618b2d P b14ea856810949aab8a8c26fecd268d8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 16:24:19.121662 17197 ts_tablet_manager.cc:1905] T 8d6b7976043647df81bc2b8346618b2d P 9057b77753d449138fa49b324f9b6929: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 16:24:19.123903 17197 ts_tablet_manager.cc:1918] T 8d6b7976043647df81bc2b8346618b2d P 9057b77753d449138fa49b324f9b6929: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 16:24:19.123976 17197 log.cc:1196] T 8d6b7976043647df81bc2b8346618b2d P 9057b77753d449138fa49b324f9b6929: Deleting WAL directory at /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786292417722232-21710-0/minicluster-data/ts-1/wal/wals/8d6b7976043647df81bc2b8346618b2d
I20260809 16:24:19.124253 17197 ts_tablet_manager.cc:1939] T 8d6b7976043647df81bc2b8346618b2d P 9057b77753d449138fa49b324f9b6929: Deleting consensus metadata
I20260809 16:24:19.125120 16439 catalog_manager.cc:4841] TS 9057b77753d449138fa49b324f9b6929 (127.21.51.130:45997): tablet 8d6b7976043647df81bc2b8346618b2d (table test-table [id=423cb63b64bf4835bd9aebd58c0512ed]) successfully deleted
I20260809 16:24:19.126972 17198 raft_consensus.cc:2258] T 8d6b7976043647df81bc2b8346618b2d P 0f0d3a27019f41c6bad1daf784349767 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 16:24:19.127499 17198 ts_tablet_manager.cc:1905] T 8d6b7976043647df81bc2b8346618b2d P 0f0d3a27019f41c6bad1daf784349767: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 16:24:19.128779 17198 ts_tablet_manager.cc:1918] T 8d6b7976043647df81bc2b8346618b2d P 0f0d3a27019f41c6bad1daf784349767: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 16:24:19.128849 17198 log.cc:1196] T 8d6b7976043647df81bc2b8346618b2d P 0f0d3a27019f41c6bad1daf784349767: Deleting WAL directory at /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786292417722232-21710-0/minicluster-data/ts-2/wal/wals/8d6b7976043647df81bc2b8346618b2d
I20260809 16:24:19.129106 17198 ts_tablet_manager.cc:1939] T 8d6b7976043647df81bc2b8346618b2d P 0f0d3a27019f41c6bad1daf784349767: Deleting consensus metadata
I20260809 16:24:19.129506 16439 catalog_manager.cc:4841] TS 0f0d3a27019f41c6bad1daf784349767 (127.21.51.131:45309): tablet 8d6b7976043647df81bc2b8346618b2d (table test-table [id=423cb63b64bf4835bd9aebd58c0512ed]) successfully deleted
W20260809 16:24:19.131716 16436 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.21.51.129:33847 (error 108)
I20260809 16:24:19.132019 21710 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrNqUCn/build/release/bin/kudu with pid 16760
I20260809 16:24:19.132225 17164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:19.132068 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.129:33847 (local address 127.0.0.1:53016)
0809 16:24:19.132134 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0809 16:24:19.132184 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.51.129:33847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260809 16:24:19.138486 16436 catalog_manager.cc:4584] TS b14ea856810949aab8a8c26fecd268d8 (127.21.51.129:33847): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 8d6b7976043647df81bc2b8346618b2d: Network error: Client connection negotiation failed: client connection to 127.21.51.129:33847: connect: Connection refused (error 111)
I20260809 16:24:19.146205 21710 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrNqUCn/build/release/bin/kudu with pid 16956
I20260809 16:24:19.157039 21710 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrNqUCn/build/release/bin/kudu with pid 16413
2026-08-09T16:24:19Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (727 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786292417722232-21710-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 16:24:19 dist-test-slave-s11t krb5kdc[17208](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 16:24:19 dist-test-slave-s11t krb5kdc[17208](info): set up 2 sockets
Aug 09 16:24:19 dist-test-slave-s11t krb5kdc[17208](info): commencing operation
krb5kdc: starting...
W20260809 16:24:20.412216 21710 mini_kdc.cc:121] Time spent starting KDC: real 1.231s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786292417722232-21710-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786292417722232-21710-0/krb5kdc/test-user.keytab.
Aug 09 16:24:20 dist-test-slave-s11t krb5kdc[17208](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292660, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T16:24:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T16:24:20Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.51.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.51.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.51.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.190.keytab.
Entry for principal kudu/127.21.51.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.190.keytab.
I20260809 16:24:20.535422 21710 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrNqUCn/build/release/bin/kudu
/tmp/dist-test-taskrNqUCn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786292417722232-21710-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786292417722232-21710-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786292417722232-21710-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786292417722232-21710-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.51.190:41521
--webserver_interface=127.21.51.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.190.keytab
--principal=kudu/127.21.51.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.51.148:44077
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.51.190:41521
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786292417722232-21710-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786292417722232-21710-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786292417722232-21710-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:24:20.618892 17228 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:24:20.619132 17228 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:24:20.619171 17228 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:24:20.620312 17228 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 16:24:20.620381 17228 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:24:20.620406 17228 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:24:20.620443 17228 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 16:24:20.620472 17228 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 16:24:20.622006 17228 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.51.148:44077
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786292417722232-21710-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786292417722232-21710-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.51.190:41521
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.190.keytab
--principal=kudu/127.21.51.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.51.190:41521
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786292417722232-21710-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.51.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17228
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786292417722232-21710-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:24:20.622287 17228 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:24:20.622529 17228 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:24:20.627490 17238 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:24:20.627504 17234 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:24:20.627490 17235 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:24:20.627745 17228 server_base.cc:1034] running on GCE node
I20260809 16:24:20.627983 17228 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:24:20.628271 17228 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:24:20.629449 17228 hybrid_clock.cc:648] HybridClock initialized: now 1786292660629441 us; error 58 us; skew 500 ppm
Aug 09 16:24:20 dist-test-slave-s11t krb5kdc[17208](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292660, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.51.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:24:20.632082 17228 init.cc:377] Logged in from keytab as kudu/127.21.51.190@KRBTEST.COM (short username kudu)
I20260809 16:24:20.632472 17228 webserver.cc:460] Webserver started at http://127.21.51.190:38841/ using document root <none> and password file <none>
I20260809 16:24:20.632653 17228 fs_manager.cc:359] Metadata directory not provided
I20260809 16:24:20.632751 17228 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:24:20.632872 17228 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:24:20.633937 17228 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786292417722232-21710-0/minicluster-data/master-0/data/instance:
uuid: "4963572a4a194f24b98c7656e9ed70a6"
format_stamp: "Formatted at 2026-08-09 16:24:20 on dist-test-slave-s11t"
server_key: "6ea086ba5d8ce970412e29c17459bada"
server_key_iv: "0e3c4ee909aefd667ec51acec395ddde"
server_key_version: "encryptionkey@0"
I20260809 16:24:20.634255 17228 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786292417722232-21710-0/minicluster-data/master-0/wal/instance:
uuid: "4963572a4a194f24b98c7656e9ed70a6"
format_stamp: "Formatted at 2026-08-09 16:24:20 on dist-test-slave-s11t"
server_key: "6ea086ba5d8ce970412e29c17459bada"
server_key_iv: "0e3c4ee909aefd667ec51acec395ddde"
server_key_version: "encryptionkey@0"
I20260809 16:24:20.635569 17228 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 16:24:20.636417 17251 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:24:20.636660 17228 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 16:24:20.636850 17228 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786292417722232-21710-0/minicluster-data/master-0/data,/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786292417722232-21710-0/minicluster-data/master-0/wal
uuid: "4963572a4a194f24b98c7656e9ed70a6"
format_stamp: "Formatted at 2026-08-09 16:24:20 on dist-test-slave-s11t"
server_key: "6ea086ba5d8ce970412e29c17459bada"
server_key_iv: "0e3c4ee909aefd667ec51acec395ddde"
server_key_version: "encryptionkey@0"
I20260809 16:24:20.636935 17228 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786292417722232-21710-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786292417722232-21710-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786292417722232-21710-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:24:20.651306 17228 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:24:20.653258 17228 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:24:20.653426 17228 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:24:20.657639 17228 rpc_server.cc:307] RPC server started. Bound to: 127.21.51.190:41521
I20260809 16:24:20.657670 17323 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.51.190:41521 every 8 connection(s)
I20260809 16:24:20.658021 17228 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786292417722232-21710-0/minicluster-data/master-0/data/info.pb
I20260809 16:24:20.658892 17324 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:24:20.660943 21710 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrNqUCn/build/release/bin/kudu as pid 17228
I20260809 16:24:20.661089 21710 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786292417722232-21710-0/minicluster-data/master-0/wal/instance
I20260809 16:24:20.661307 21710 external_mini_cluster.cc:1428] Setting key 448aac9077a6c35a6b0403eb5e7390f0
I20260809 16:24:20.661456 17324 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4963572a4a194f24b98c7656e9ed70a6: Bootstrap starting.
I20260809 16:24:20.662254 17324 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4963572a4a194f24b98c7656e9ed70a6: Neither blocks nor log segments found. Creating new log.
I20260809 16:24:20.662576 17324 log.cc:824] T 00000000000000000000000000000000 P 4963572a4a194f24b98c7656e9ed70a6: Log is configured to *not* fsync() on all Append() calls
I20260809 16:24:20.663285 17324 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4963572a4a194f24b98c7656e9ed70a6: No bootstrap required, opened a new log
I20260809 16:24:20.665122 17324 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4963572a4a194f24b98c7656e9ed70a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4963572a4a194f24b98c7656e9ed70a6" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 41521 } }
I20260809 16:24:20.665278 17324 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4963572a4a194f24b98c7656e9ed70a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:24:20.665343 17324 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4963572a4a194f24b98c7656e9ed70a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4963572a4a194f24b98c7656e9ed70a6, State: Initialized, Role: FOLLOWER
I20260809 16:24:20.665503 17324 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4963572a4a194f24b98c7656e9ed70a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4963572a4a194f24b98c7656e9ed70a6" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 41521 } }
I20260809 16:24:20.665601 17324 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4963572a4a194f24b98c7656e9ed70a6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 16:24:20.665647 17324 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4963572a4a194f24b98c7656e9ed70a6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 16:24:20.665694 17324 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4963572a4a194f24b98c7656e9ed70a6 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:24:20.666224 17324 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4963572a4a194f24b98c7656e9ed70a6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4963572a4a194f24b98c7656e9ed70a6" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 41521 } }
I20260809 16:24:20.666347 17324 leader_election.cc:302] T 00000000000000000000000000000000 P 4963572a4a194f24b98c7656e9ed70a6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4963572a4a194f24b98c7656e9ed70a6; no voters:
I20260809 16:24:20.666523 17324 leader_election.cc:288] T 00000000000000000000000000000000 P 4963572a4a194f24b98c7656e9ed70a6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 16:24:20.666675 17329 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4963572a4a194f24b98c7656e9ed70a6 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:24:20.666790 17324 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4963572a4a194f24b98c7656e9ed70a6 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 16:24:20.666899 17329 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4963572a4a194f24b98c7656e9ed70a6 [term 1 LEADER]: Becoming Leader. State: Replica: 4963572a4a194f24b98c7656e9ed70a6, State: Running, Role: LEADER
I20260809 16:24:20.667035 17329 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4963572a4a194f24b98c7656e9ed70a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4963572a4a194f24b98c7656e9ed70a6" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 41521 } }
I20260809 16:24:20.667433 17330 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4963572a4a194f24b98c7656e9ed70a6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4963572a4a194f24b98c7656e9ed70a6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4963572a4a194f24b98c7656e9ed70a6" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 41521 } } }
I20260809 16:24:20.667506 17330 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4963572a4a194f24b98c7656e9ed70a6 [sys.catalog]: This master's current role is: LEADER
I20260809 16:24:20.667673 17333 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4963572a4a194f24b98c7656e9ed70a6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4963572a4a194f24b98c7656e9ed70a6. Latest consensus state: current_term: 1 leader_uuid: "4963572a4a194f24b98c7656e9ed70a6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4963572a4a194f24b98c7656e9ed70a6" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 41521 } } }
I20260809 16:24:20.667753 17333 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4963572a4a194f24b98c7656e9ed70a6 [sys.catalog]: This master's current role is: LEADER
I20260809 16:24:20.668197 17343 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
Aug 09 16:24:20 dist-test-slave-s11t krb5kdc[17208](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292660, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.51.190@KRBTEST.COM
I20260809 16:24:20.668905 17343 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 16:24:20.670585 17343 catalog_manager.cc:1347] Generated new cluster ID: 5ba6dded59664e38bb50c23a73ddfc76
I20260809 16:24:20.670643 17343 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 16:24:20.672528 17327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:20.661872 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59522 (local address 127.21.51.190:41521)
0809 16:24:20.662068 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0809 16:24:20.662078 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 16:24:20.662118 (+ 40us) server_negotiation.cc:407] Connection header received
0809 16:24:20.662619 (+ 501us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:24:20.662626 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:24:20.662717 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:24:20.662877 (+ 160us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:24:20.663587 (+ 710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:24:20.664488 (+ 901us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:24:20.665322 (+ 834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:24:20.665923 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:24:20.666046 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:24:20.669112 (+ 3066us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:24:20.669116 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:24:20.669116 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:24:20.669128 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:24:20.671023 (+ 1895us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:24:20.671387 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:24:20.671390 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:24:20.671390 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:24:20.671491 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:24:20.671705 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:24:20.671706 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:24:20.671707 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:24:20.671949 (+ 242us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:24:20.672120 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0809 16:24:20.672203 (+ 83us) server_negotiation.cc:299] Negotiation successful
0809 16:24:20.672341 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":149,"thread_start_us":103,"threads_started":1}
I20260809 16:24:20.689090 17343 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 16:24:20.689688 17343 catalog_manager.cc:1504] Loading token signing keys...
I20260809 16:24:20.695758 17343 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4963572a4a194f24b98c7656e9ed70a6: Generated new TSK 0
I20260809 16:24:20.695945 17343 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.21.51.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.51.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.51.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.129.keytab.
Entry for principal kudu/127.21.51.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.129.keytab.
I20260809 16:24:20.731405 21710 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrNqUCn/build/release/bin/kudu
/tmp/dist-test-taskrNqUCn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786292417722232-21710-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786292417722232-21710-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786292417722232-21710-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786292417722232-21710-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.51.129:0
--local_ip_for_outbound_sockets=127.21.51.129
--webserver_interface=127.21.51.129
--webserver_port=0
--tserver_master_addrs=127.21.51.190:41521
--keytab_file=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.129.keytab
--principal=kudu/127.21.51.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.51.148:44077
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786292417722232-21710-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786292417722232-21710-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786292417722232-21710-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:24:20.814253 17363 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:24:20.814489 17363 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:24:20.814527 17363 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:24:20.815634 17363 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:24:20.815699 17363 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:24:20.815742 17363 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.51.129
I20260809 16:24:20.817198 17363 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.51.148:44077
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786292417722232-21710-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786292417722232-21710-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.129.keytab
--principal=kudu/127.21.51.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.51.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786292417722232-21710-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.51.129
--webserver_port=0
--tserver_master_addrs=127.21.51.190:41521
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17363
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.51.129
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786292417722232-21710-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:24:20.817462 17363 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:24:20.817698 17363 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:24:20.822628 17373 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:24:20.822645 17371 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:24:20.822866 17363 server_base.cc:1034] running on GCE node
W20260809 16:24:20.822608 17370 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:24:20.823138 17363 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:24:20.823391 17363 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:24:20.824571 17363 hybrid_clock.cc:648] HybridClock initialized: now 1786292660824560 us; error 50 us; skew 500 ppm
Aug 09 16:24:20 dist-test-slave-s11t krb5kdc[17208](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292660, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.51.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:24:20.827306 17363 init.cc:377] Logged in from keytab as kudu/127.21.51.129@KRBTEST.COM (short username kudu)
I20260809 16:24:20.827701 17363 webserver.cc:460] Webserver started at http://127.21.51.129:36401/ using document root <none> and password file <none>
I20260809 16:24:20.827879 17363 fs_manager.cc:359] Metadata directory not provided
I20260809 16:24:20.827919 17363 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:24:20.828006 17363 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:24:20.829166 17363 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786292417722232-21710-0/minicluster-data/ts-0/data/instance:
uuid: "673c507686504b4faccf602508d3b822"
format_stamp: "Formatted at 2026-08-09 16:24:20 on dist-test-slave-s11t"
server_key: "0deb480d8fd3b70bba1a923215f8dae7"
server_key_iv: "54b2ade54cc63e0eaa816849805c4af5"
server_key_version: "encryptionkey@0"
I20260809 16:24:20.829486 17363 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786292417722232-21710-0/minicluster-data/ts-0/wal/instance:
uuid: "673c507686504b4faccf602508d3b822"
format_stamp: "Formatted at 2026-08-09 16:24:20 on dist-test-slave-s11t"
server_key: "0deb480d8fd3b70bba1a923215f8dae7"
server_key_iv: "54b2ade54cc63e0eaa816849805c4af5"
server_key_version: "encryptionkey@0"
I20260809 16:24:20.831073 17363 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 16:24:20.832001 17382 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:24:20.832190 17363 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:24:20.832297 17363 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786292417722232-21710-0/minicluster-data/ts-0/data,/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786292417722232-21710-0/minicluster-data/ts-0/wal
uuid: "673c507686504b4faccf602508d3b822"
format_stamp: "Formatted at 2026-08-09 16:24:20 on dist-test-slave-s11t"
server_key: "0deb480d8fd3b70bba1a923215f8dae7"
server_key_iv: "54b2ade54cc63e0eaa816849805c4af5"
server_key_version: "encryptionkey@0"
I20260809 16:24:20.832353 17363 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786292417722232-21710-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786292417722232-21710-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786292417722232-21710-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:24:20.849263 17363 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:24:20.851603 17363 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:24:20.851729 17363 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:24:20.851936 17363 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:24:20.852274 17363 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:24:20.852311 17363 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:24:20.852334 17363 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:24:20.852383 17363 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:24:20.858196 17363 rpc_server.cc:307] RPC server started. Bound to: 127.21.51.129:39649
I20260809 16:24:20.858280 17558 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.51.129:39649 every 8 connection(s)
I20260809 16:24:20.858615 17363 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786292417722232-21710-0/minicluster-data/ts-0/data/info.pb
Aug 09 16:24:20 dist-test-slave-s11t krb5kdc[17208](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292660, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.51.129@KRBTEST.COM for kudu/127.21.51.190@KRBTEST.COM
I20260809 16:24:20.867014 21710 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrNqUCn/build/release/bin/kudu as pid 17363
I20260809 16:24:20.867277 21710 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786292417722232-21710-0/minicluster-data/ts-0/wal/instance
I20260809 16:24:20.867560 21710 external_mini_cluster.cc:1428] Setting key 27c16227a5f99d219030b8183fd2f0cd
I20260809 16:24:20.869266 17327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:20.859361 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.51.129:51225 (local address 127.21.51.190:41521)
0809 16:24:20.859408 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0809 16:24:20.859410 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:24:20.859636 (+ 226us) server_negotiation.cc:407] Connection header received
0809 16:24:20.860087 (+ 451us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:24:20.860089 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:24:20.860101 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:24:20.860173 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:24:20.861358 (+ 1185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:24:20.861903 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:24:20.862914 (+ 1011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:24:20.863142 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:24:20.863207 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:24:20.865494 (+ 2287us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:24:20.865496 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:24:20.865496 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:24:20.865503 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:24:20.867471 (+ 1968us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:24:20.868037 (+ 566us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:24:20.868038 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:24:20.868038 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:24:20.868106 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:24:20.868487 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:24:20.868488 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:24:20.868488 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:24:20.868616 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:24:20.868732 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0809 16:24:20.869112 (+ 380us) server_negotiation.cc:299] Negotiation successful
0809 16:24:20.869196 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260809 16:24:20.869684 17563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:20.859309 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:41521 (local address 127.21.51.129:51225)
0809 16:24:20.859479 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0809 16:24:20.859522 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0809 16:24:20.859991 (+ 469us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:24:20.860311 (+ 320us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:24:20.860314 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:24:20.861027 (+ 713us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:24:20.861228 (+ 201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:24:20.861233 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:24:20.861999 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:24:20.862000 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:24:20.862807 (+ 807us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:24:20.862809 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:24:20.863225 (+ 416us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:24:20.863226 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:24:20.863329 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:24:20.863800 (+ 471us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:24:20.863808 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:24:20.865374 (+ 1566us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:24:20.867552 (+ 2178us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:24:20.867561 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:24:20.867563 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:24:20.867941 (+ 378us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:24:20.868196 (+ 255us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:24:20.868197 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:24:20.868197 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:24:20.868392 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:24:20.868730 (+ 338us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:24:20.868738 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:24:20.868929 (+ 191us) client_negotiation.cc:769] Sending connection context
0809 16:24:20.869080 (+ 151us) client_negotiation.cc:240] Negotiation successful
0809 16:24:20.869295 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":91,"threads_started":1}
I20260809 16:24:20.870905 17560 heartbeater.cc:347] Connected to a master server at 127.21.51.190:41521
I20260809 16:24:20.871023 17560 heartbeater.cc:464] Registering TS with master...
I20260809 16:24:20.871218 17560 heartbeater.cc:511] Master 127.21.51.190:41521 requested a full tablet report, sending...
I20260809 16:24:20.871851 17275 ts_manager.cc:194] Registered new tserver with Master: 673c507686504b4faccf602508d3b822 (127.21.51.129:39649)
I20260809 16:24:20.872917 17275 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.51.129@KRBTEST.COM'} at 127.21.51.129:51225
WARNING: no policy specified for kudu/127.21.51.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.51.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.51.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.130.keytab.
Entry for principal kudu/127.21.51.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.130.keytab.
I20260809 16:24:20.896242 21710 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrNqUCn/build/release/bin/kudu
/tmp/dist-test-taskrNqUCn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786292417722232-21710-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786292417722232-21710-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786292417722232-21710-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786292417722232-21710-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.51.130:0
--local_ip_for_outbound_sockets=127.21.51.130
--webserver_interface=127.21.51.130
--webserver_port=0
--tserver_master_addrs=127.21.51.190:41521
--keytab_file=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.130.keytab
--principal=kudu/127.21.51.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.51.148:44077
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786292417722232-21710-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786292417722232-21710-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786292417722232-21710-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:24:20.978948 17566 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:24:20.979144 17566 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:24:20.979166 17566 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:24:20.980229 17566 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:24:20.980265 17566 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:24:20.980290 17566 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.51.130
I20260809 16:24:20.981739 17566 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.51.148:44077
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786292417722232-21710-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786292417722232-21710-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.130.keytab
--principal=kudu/127.21.51.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.51.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786292417722232-21710-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.51.130
--webserver_port=0
--tserver_master_addrs=127.21.51.190:41521
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17566
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.51.130
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786292417722232-21710-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:24:20.981932 17566 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:24:20.982103 17566 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:24:20.987171 17571 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:24:20.987172 17574 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:24:20.987254 17572 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:24:20.987581 17566 server_base.cc:1034] running on GCE node
I20260809 16:24:20.987772 17566 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:24:20.987995 17566 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:24:20.989135 17566 hybrid_clock.cc:648] HybridClock initialized: now 1786292660989123 us; error 37 us; skew 500 ppm
Aug 09 16:24:20 dist-test-slave-s11t krb5kdc[17208](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292660, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.51.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:24:20.991659 17566 init.cc:377] Logged in from keytab as kudu/127.21.51.130@KRBTEST.COM (short username kudu)
I20260809 16:24:20.992113 17566 webserver.cc:460] Webserver started at http://127.21.51.130:41965/ using document root <none> and password file <none>
I20260809 16:24:20.992314 17566 fs_manager.cc:359] Metadata directory not provided
I20260809 16:24:20.992355 17566 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:24:20.992444 17566 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:24:20.993494 17566 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786292417722232-21710-0/minicluster-data/ts-1/data/instance:
uuid: "a96001122a9a40b8a7f65d6e9510177e"
format_stamp: "Formatted at 2026-08-09 16:24:20 on dist-test-slave-s11t"
server_key: "e99491d2d7ecb0becca339b5a40ef48c"
server_key_iv: "44b3b9f8f5cd58e7abb817acc1ee8ea4"
server_key_version: "encryptionkey@0"
I20260809 16:24:20.993800 17566 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786292417722232-21710-0/minicluster-data/ts-1/wal/instance:
uuid: "a96001122a9a40b8a7f65d6e9510177e"
format_stamp: "Formatted at 2026-08-09 16:24:20 on dist-test-slave-s11t"
server_key: "e99491d2d7ecb0becca339b5a40ef48c"
server_key_iv: "44b3b9f8f5cd58e7abb817acc1ee8ea4"
server_key_version: "encryptionkey@0"
I20260809 16:24:20.995046 17566 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 16:24:20.995712 17583 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:24:20.995869 17566 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 16:24:20.995960 17566 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786292417722232-21710-0/minicluster-data/ts-1/data,/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786292417722232-21710-0/minicluster-data/ts-1/wal
uuid: "a96001122a9a40b8a7f65d6e9510177e"
format_stamp: "Formatted at 2026-08-09 16:24:20 on dist-test-slave-s11t"
server_key: "e99491d2d7ecb0becca339b5a40ef48c"
server_key_iv: "44b3b9f8f5cd58e7abb817acc1ee8ea4"
server_key_version: "encryptionkey@0"
I20260809 16:24:20.996016 17566 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786292417722232-21710-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786292417722232-21710-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786292417722232-21710-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:24:21.009123 17566 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:24:21.011451 17566 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:24:21.011564 17566 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:24:21.011747 17566 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:24:21.012069 17566 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:24:21.012105 17566 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:24:21.012128 17566 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:24:21.012141 17566 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:24:21.017820 17566 rpc_server.cc:307] RPC server started. Bound to: 127.21.51.130:42055
I20260809 16:24:21.017936 17758 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.51.130:42055 every 8 connection(s)
I20260809 16:24:21.018221 17566 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786292417722232-21710-0/minicluster-data/ts-1/data/info.pb
I20260809 16:24:21.021457 21710 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrNqUCn/build/release/bin/kudu as pid 17566
I20260809 16:24:21.021582 21710 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786292417722232-21710-0/minicluster-data/ts-1/wal/instance
I20260809 16:24:21.021819 21710 external_mini_cluster.cc:1428] Setting key c3bebbf8fdc69a94e689139f8e24dea6
Aug 09 16:24:21 dist-test-slave-s11t krb5kdc[17208](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292660, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.51.130@KRBTEST.COM for kudu/127.21.51.190@KRBTEST.COM
I20260809 16:24:21.030102 17765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:21.018905 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:41521 (local address 127.21.51.130:37849)
0809 16:24:21.019140 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0809 16:24:21.019190 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0809 16:24:21.019755 (+ 565us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:24:21.020082 (+ 327us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:24:21.020087 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:24:21.020806 (+ 719us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:24:21.021011 (+ 205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:24:21.021017 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:24:21.022139 (+ 1122us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:24:21.022140 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:24:21.023378 (+ 1238us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:24:21.023382 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:24:21.023744 (+ 362us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:24:21.023745 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:24:21.023913 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:24:21.024420 (+ 507us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:24:21.024431 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:24:21.026209 (+ 1778us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:24:21.028503 (+ 2294us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:24:21.028509 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:24:21.028511 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:24:21.028829 (+ 318us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:24:21.029069 (+ 240us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:24:21.029070 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:24:21.029070 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:24:21.029177 (+ 107us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:24:21.029544 (+ 367us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:24:21.029549 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:24:21.029671 (+ 122us) client_negotiation.cc:769] Sending connection context
0809 16:24:21.029799 (+ 128us) client_negotiation.cc:240] Negotiation successful
0809 16:24:21.029929 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":96,"threads_started":1}
I20260809 16:24:21.030148 17327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:21.018975 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.51.130:37849 (local address 127.21.51.190:41521)
0809 16:24:21.019121 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0809 16:24:21.019124 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 16:24:21.019314 (+ 190us) server_negotiation.cc:407] Connection header received
0809 16:24:21.019914 (+ 600us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:24:21.019916 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:24:21.019929 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:24:21.019999 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:24:21.021117 (+ 1118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:24:21.021995 (+ 878us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:24:21.023466 (+ 1471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:24:21.023654 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:24:21.023712 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:24:21.026332 (+ 2620us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:24:21.026334 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:24:21.026334 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:24:21.026343 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:24:21.028405 (+ 2062us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:24:21.028920 (+ 515us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:24:21.028922 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:24:21.028922 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:24:21.028990 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:24:21.029291 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:24:21.029292 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:24:21.029293 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:24:21.029466 (+ 173us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:24:21.029547 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0809 16:24:21.029880 (+ 333us) server_negotiation.cc:299] Negotiation successful
0809 16:24:21.029993 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":109}
I20260809 16:24:21.030704 17761 heartbeater.cc:347] Connected to a master server at 127.21.51.190:41521
I20260809 16:24:21.030795 17761 heartbeater.cc:464] Registering TS with master...
I20260809 16:24:21.031001 17761 heartbeater.cc:511] Master 127.21.51.190:41521 requested a full tablet report, sending...
I20260809 16:24:21.031463 17275 ts_manager.cc:194] Registered new tserver with Master: a96001122a9a40b8a7f65d6e9510177e (127.21.51.130:42055)
I20260809 16:24:21.032068 17275 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.51.130@KRBTEST.COM'} at 127.21.51.130:37849
WARNING: no policy specified for kudu/127.21.51.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.51.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.51.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.131.keytab.
Entry for principal kudu/127.21.51.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.131.keytab.
I20260809 16:24:21.052343 21710 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrNqUCn/build/release/bin/kudu
/tmp/dist-test-taskrNqUCn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786292417722232-21710-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786292417722232-21710-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786292417722232-21710-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786292417722232-21710-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.51.131:0
--local_ip_for_outbound_sockets=127.21.51.131
--webserver_interface=127.21.51.131
--webserver_port=0
--tserver_master_addrs=127.21.51.190:41521
--keytab_file=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.131.keytab
--principal=kudu/127.21.51.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.51.148:44077
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786292417722232-21710-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786292417722232-21710-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786292417722232-21710-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:24:21.138692 17770 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:24:21.138895 17770 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:24:21.138919 17770 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:24:21.139999 17770 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:24:21.140031 17770 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:24:21.140056 17770 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.51.131
I20260809 16:24:21.141487 17770 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.51.148:44077
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786292417722232-21710-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786292417722232-21710-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.131.keytab
--principal=kudu/127.21.51.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.51.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786292417722232-21710-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.51.131
--webserver_port=0
--tserver_master_addrs=127.21.51.190:41521
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17770
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.51.131
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786292417722232-21710-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:24:21.141665 17770 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:24:21.141839 17770 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:24:21.147015 17778 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:24:21.147029 17776 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:24:21.147070 17775 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:24:21.147065 17770 server_base.cc:1034] running on GCE node
I20260809 16:24:21.147442 17770 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:24:21.147727 17770 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:24:21.148929 17770 hybrid_clock.cc:648] HybridClock initialized: now 1786292661148911 us; error 47 us; skew 500 ppm
Aug 09 16:24:21 dist-test-slave-s11t krb5kdc[17208](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292661, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.51.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:24:21.151479 17770 init.cc:377] Logged in from keytab as kudu/127.21.51.131@KRBTEST.COM (short username kudu)
I20260809 16:24:21.151849 17770 webserver.cc:460] Webserver started at http://127.21.51.131:41497/ using document root <none> and password file <none>
I20260809 16:24:21.152099 17770 fs_manager.cc:359] Metadata directory not provided
I20260809 16:24:21.152171 17770 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:24:21.152328 17770 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:24:21.153469 17770 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786292417722232-21710-0/minicluster-data/ts-2/data/instance:
uuid: "99915d8b5bc04a8d8ff2e03ec84130a4"
format_stamp: "Formatted at 2026-08-09 16:24:21 on dist-test-slave-s11t"
server_key: "4fdc6858a6a88a2c00046aa487e5eaa4"
server_key_iv: "bde2188e66d6bc31134babef2e90c91b"
server_key_version: "encryptionkey@0"
I20260809 16:24:21.153854 17770 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786292417722232-21710-0/minicluster-data/ts-2/wal/instance:
uuid: "99915d8b5bc04a8d8ff2e03ec84130a4"
format_stamp: "Formatted at 2026-08-09 16:24:21 on dist-test-slave-s11t"
server_key: "4fdc6858a6a88a2c00046aa487e5eaa4"
server_key_iv: "bde2188e66d6bc31134babef2e90c91b"
server_key_version: "encryptionkey@0"
I20260809 16:24:21.155198 17770 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 16:24:21.155970 17788 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:24:21.156160 17770 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:24:21.156265 17770 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786292417722232-21710-0/minicluster-data/ts-2/data,/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786292417722232-21710-0/minicluster-data/ts-2/wal
uuid: "99915d8b5bc04a8d8ff2e03ec84130a4"
format_stamp: "Formatted at 2026-08-09 16:24:21 on dist-test-slave-s11t"
server_key: "4fdc6858a6a88a2c00046aa487e5eaa4"
server_key_iv: "bde2188e66d6bc31134babef2e90c91b"
server_key_version: "encryptionkey@0"
I20260809 16:24:21.156347 17770 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786292417722232-21710-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786292417722232-21710-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786292417722232-21710-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:24:21.170246 17770 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:24:21.176440 17770 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:24:21.176614 17770 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:24:21.176883 17770 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:24:21.177227 17770 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:24:21.177285 17770 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:24:21.177327 17770 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:24:21.177352 17770 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:24:21.183485 17770 rpc_server.cc:307] RPC server started. Bound to: 127.21.51.131:33953
I20260809 16:24:21.183590 17956 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.51.131:33953 every 8 connection(s)
I20260809 16:24:21.183970 17770 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786292417722232-21710-0/minicluster-data/ts-2/data/info.pb
I20260809 16:24:21.187826 21710 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrNqUCn/build/release/bin/kudu as pid 17770
I20260809 16:24:21.187973 21710 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786292417722232-21710-0/minicluster-data/ts-2/wal/instance
I20260809 16:24:21.188167 21710 external_mini_cluster.cc:1428] Setting key 65f642728c82a0062a2e408eadcfc08e
Aug 09 16:24:21 dist-test-slave-s11t krb5kdc[17208](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292661, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.51.131@KRBTEST.COM for kudu/127.21.51.190@KRBTEST.COM
I20260809 16:24:21.193953 17327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:21.184864 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.51.131:48847 (local address 127.21.51.190:41521)
0809 16:24:21.184913 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0809 16:24:21.184915 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:24:21.185115 (+ 200us) server_negotiation.cc:407] Connection header received
0809 16:24:21.185619 (+ 504us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:24:21.185620 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:24:21.185631 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:24:21.185745 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:24:21.186678 (+ 933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:24:21.187226 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:24:21.188586 (+ 1360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:24:21.188889 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:24:21.188964 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:24:21.191081 (+ 2117us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:24:21.191083 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:24:21.191083 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:24:21.191091 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:24:21.192435 (+ 1344us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:24:21.192938 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:24:21.192939 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:24:21.192940 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:24:21.193006 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:24:21.193272 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:24:21.193273 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:24:21.193273 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:24:21.193402 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:24:21.193480 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0809 16:24:21.193761 (+ 281us) server_negotiation.cc:299] Negotiation successful
0809 16:24:21.193850 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260809 16:24:21.194125 17960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:21.184830 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:41521 (local address 127.21.51.131:48847)
0809 16:24:21.184986 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0809 16:24:21.185021 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0809 16:24:21.185493 (+ 472us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:24:21.185797 (+ 304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:24:21.185800 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:24:21.186367 (+ 567us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:24:21.186572 (+ 205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:24:21.186576 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:24:21.187319 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:24:21.187320 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:24:21.188495 (+ 1175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:24:21.188497 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:24:21.188983 (+ 486us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:24:21.188984 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:24:21.189127 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:24:21.189602 (+ 475us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:24:21.189610 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:24:21.190980 (+ 1370us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:24:21.192521 (+ 1541us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:24:21.192528 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:24:21.192531 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:24:21.192841 (+ 310us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:24:21.193083 (+ 242us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:24:21.193084 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:24:21.193084 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:24:21.193196 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:24:21.193470 (+ 274us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:24:21.193477 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:24:21.193621 (+ 144us) client_negotiation.cc:769] Sending connection context
0809 16:24:21.193727 (+ 106us) client_negotiation.cc:240] Negotiation successful
0809 16:24:21.193889 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":92,"threads_started":1}
I20260809 16:24:21.194561 17957 heartbeater.cc:347] Connected to a master server at 127.21.51.190:41521
I20260809 16:24:21.194666 17957 heartbeater.cc:464] Registering TS with master...
I20260809 16:24:21.194856 17957 heartbeater.cc:511] Master 127.21.51.190:41521 requested a full tablet report, sending...
I20260809 16:24:21.195276 17275 ts_manager.cc:194] Registered new tserver with Master: 99915d8b5bc04a8d8ff2e03ec84130a4 (127.21.51.131:33953)
I20260809 16:24:21.195876 17275 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.51.131@KRBTEST.COM'} at 127.21.51.131:48847
I20260809 16:24:21.200357 21710 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 16:24:21.208747 17327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:21.201905 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59538 (local address 127.21.51.190:41521)
0809 16:24:21.201995 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0809 16:24:21.201998 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 16:24:21.202099 (+ 101us) server_negotiation.cc:407] Connection header received
0809 16:24:21.202171 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:24:21.202172 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:24:21.202183 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:24:21.202245 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:24:21.202908 (+ 663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:24:21.203454 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:24:21.204256 (+ 802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:24:21.204485 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:24:21.204571 (+ 86us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:24:21.206022 (+ 1451us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:24:21.206024 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:24:21.206024 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:24:21.206033 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:24:21.207267 (+ 1234us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:24:21.207739 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:24:21.207741 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:24:21.207742 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:24:21.207810 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:24:21.208062 (+ 252us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:24:21.208064 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:24:21.208064 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:24:21.208234 (+ 170us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:24:21.208370 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0809 16:24:21.208502 (+ 132us) server_negotiation.cc:299] Negotiation successful
0809 16:24:21.208574 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260809 16:24:21.210582 17275 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59538:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 16:24:21.212337 17275 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 16:24:21.219581 17973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:21.214980 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.130:42055 (local address 127.0.0.1:36752)
0809 16:24:21.215428 (+ 448us) negotiation.cc:107] Waiting for socket to connect
0809 16:24:21.215441 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 16:24:21.215503 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:24:21.215837 (+ 334us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:24:21.215838 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:24:21.215840 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:24:21.215907 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:24:21.215909 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:24:21.217003 (+ 1094us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:24:21.217004 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:24:21.218456 (+ 1452us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:24:21.218459 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:24:21.219244 (+ 785us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:24:21.219245 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:24:21.219344 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:24:21.219361 (+ 17us) client_negotiation.cc:769] Sending connection context
0809 16:24:21.219437 (+ 76us) client_negotiation.cc:240] Negotiation successful
0809 16:24:21.219500 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":424,"mutex_wait_us":37,"thread_start_us":85,"threads_started":1}
I20260809 16:24:21.219645 17975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:21.215227 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36752 (local address 127.21.51.130:42055)
0809 16:24:21.215476 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0809 16:24:21.215480 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 16:24:21.215492 (+ 12us) server_negotiation.cc:407] Connection header received
0809 16:24:21.215618 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:24:21.215622 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:24:21.215645 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:24:21.215737 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:24:21.215995 (+ 258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:24:21.216893 (+ 898us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:24:21.218580 (+ 1687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:24:21.219147 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:24:21.219244 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:24:21.219309 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0809 16:24:21.219506 (+ 197us) server_negotiation.cc:299] Negotiation successful
0809 16:24:21.219553 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":208,"thread_start_us":106,"threads_started":1}
I20260809 16:24:21.219815 17971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:21.214953 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.129:39649 (local address 127.0.0.1:34400)
0809 16:24:21.215081 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0809 16:24:21.215123 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0809 16:24:21.215198 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:24:21.215671 (+ 473us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:24:21.215675 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:24:21.215678 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:24:21.215789 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:24:21.215791 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:24:21.217359 (+ 1568us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:24:21.217360 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:24:21.218583 (+ 1223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:24:21.218585 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:24:21.219542 (+ 957us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:24:21.219543 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:24:21.219616 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:24:21.219621 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 16:24:21.219657 (+ 36us) client_negotiation.cc:240] Negotiation successful
0809 16:24:21.219704 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":100,"mutex_wait_us":24,"thread_start_us":80,"threads_started":1}
I20260809 16:24:21.220266 17974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:21.215224 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34400 (local address 127.21.51.129:39649)
0809 16:24:21.215431 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0809 16:24:21.215436 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 16:24:21.215449 (+ 13us) server_negotiation.cc:407] Connection header received
0809 16:24:21.215494 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:24:21.215497 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:24:21.215515 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:24:21.215884 (+ 369us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:24:21.215965 (+ 81us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:24:21.217242 (+ 1277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:24:21.218672 (+ 1430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:24:21.219434 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:24:21.220018 (+ 584us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:24:21.220095 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0809 16:24:21.220122 (+ 27us) server_negotiation.cc:299] Negotiation successful
0809 16:24:21.220169 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":164,"thread_start_us":108,"threads_started":1}
I20260809 16:24:21.220257 17977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:21.215424 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32926 (local address 127.21.51.131:33953)
0809 16:24:21.215613 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0809 16:24:21.215617 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 16:24:21.215755 (+ 138us) server_negotiation.cc:407] Connection header received
0809 16:24:21.215813 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:24:21.215816 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:24:21.215833 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:24:21.215910 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:24:21.216123 (+ 213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:24:21.217308 (+ 1185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:24:21.218545 (+ 1237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:24:21.219387 (+ 842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:24:21.219469 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:24:21.219537 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0809 16:24:21.220116 (+ 579us) server_negotiation.cc:299] Negotiation successful
0809 16:24:21.220169 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":143,"thread_start_us":96,"threads_started":1}
I20260809 16:24:21.220492 17972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:21.215183 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.131:33953 (local address 127.0.0.1:32926)
0809 16:24:21.215615 (+ 432us) negotiation.cc:107] Waiting for socket to connect
0809 16:24:21.215626 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 16:24:21.215681 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:24:21.215981 (+ 300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:24:21.215982 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:24:21.215984 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:24:21.216035 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:24:21.216037 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:24:21.217430 (+ 1393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:24:21.217431 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:24:21.218444 (+ 1013us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:24:21.218447 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:24:21.219570 (+ 1123us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:24:21.219571 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:24:21.219812 (+ 241us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:24:21.219817 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 16:24:21.220354 (+ 537us) client_negotiation.cc:240] Negotiation successful
0809 16:24:21.220425 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":411,"thread_start_us":97,"threads_started":1}
I20260809 16:24:21.220443 17657 tablet_service.cc:1467] Processing CreateTablet for tablet e4704d5755fe449693a506af245ba0e9 (DEFAULT_TABLE table=test-table [id=a62afb4bb73949a9971f85f4493e2989]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:24:21.220901 17657 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e4704d5755fe449693a506af245ba0e9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:24:21.221192 17456 tablet_service.cc:1467] Processing CreateTablet for tablet e4704d5755fe449693a506af245ba0e9 (DEFAULT_TABLE table=test-table [id=a62afb4bb73949a9971f85f4493e2989]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:24:21.221563 17456 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e4704d5755fe449693a506af245ba0e9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:24:21.224838 17979 tablet_bootstrap.cc:492] T e4704d5755fe449693a506af245ba0e9 P a96001122a9a40b8a7f65d6e9510177e: Bootstrap starting.
I20260809 16:24:21.225226 17982 tablet_bootstrap.cc:492] T e4704d5755fe449693a506af245ba0e9 P 673c507686504b4faccf602508d3b822: Bootstrap starting.
I20260809 16:24:21.225556 17979 tablet_bootstrap.cc:654] T e4704d5755fe449693a506af245ba0e9 P a96001122a9a40b8a7f65d6e9510177e: Neither blocks nor log segments found. Creating new log.
I20260809 16:24:21.226042 17982 tablet_bootstrap.cc:654] T e4704d5755fe449693a506af245ba0e9 P 673c507686504b4faccf602508d3b822: Neither blocks nor log segments found. Creating new log.
I20260809 16:24:21.226197 17979 log.cc:824] T e4704d5755fe449693a506af245ba0e9 P a96001122a9a40b8a7f65d6e9510177e: Log is configured to *not* fsync() on all Append() calls
I20260809 16:24:21.226179 17860 tablet_service.cc:1467] Processing CreateTablet for tablet e4704d5755fe449693a506af245ba0e9 (DEFAULT_TABLE table=test-table [id=a62afb4bb73949a9971f85f4493e2989]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:24:21.226528 17860 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e4704d5755fe449693a506af245ba0e9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:24:21.226711 17982 log.cc:824] T e4704d5755fe449693a506af245ba0e9 P 673c507686504b4faccf602508d3b822: Log is configured to *not* fsync() on all Append() calls
I20260809 16:24:21.227102 17979 tablet_bootstrap.cc:492] T e4704d5755fe449693a506af245ba0e9 P a96001122a9a40b8a7f65d6e9510177e: No bootstrap required, opened a new log
I20260809 16:24:21.227180 17979 ts_tablet_manager.cc:1397] T e4704d5755fe449693a506af245ba0e9 P a96001122a9a40b8a7f65d6e9510177e: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260809 16:24:21.227522 17982 tablet_bootstrap.cc:492] T e4704d5755fe449693a506af245ba0e9 P 673c507686504b4faccf602508d3b822: No bootstrap required, opened a new log
I20260809 16:24:21.227612 17982 ts_tablet_manager.cc:1397] T e4704d5755fe449693a506af245ba0e9 P 673c507686504b4faccf602508d3b822: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 16:24:21.229207 17979 raft_consensus.cc:348] T e4704d5755fe449693a506af245ba0e9 P a96001122a9a40b8a7f65d6e9510177e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a96001122a9a40b8a7f65d6e9510177e" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 42055 } } peers { permanent_uuid: "99915d8b5bc04a8d8ff2e03ec84130a4" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 33953 } } peers { permanent_uuid: "673c507686504b4faccf602508d3b822" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 39649 } }
I20260809 16:24:21.229395 17979 raft_consensus.cc:374] T e4704d5755fe449693a506af245ba0e9 P a96001122a9a40b8a7f65d6e9510177e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:24:21.229444 17979 raft_consensus.cc:729] T e4704d5755fe449693a506af245ba0e9 P a96001122a9a40b8a7f65d6e9510177e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a96001122a9a40b8a7f65d6e9510177e, State: Initialized, Role: FOLLOWER
I20260809 16:24:21.229357 17982 raft_consensus.cc:348] T e4704d5755fe449693a506af245ba0e9 P 673c507686504b4faccf602508d3b822 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a96001122a9a40b8a7f65d6e9510177e" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 42055 } } peers { permanent_uuid: "99915d8b5bc04a8d8ff2e03ec84130a4" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 33953 } } peers { permanent_uuid: "673c507686504b4faccf602508d3b822" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 39649 } }
I20260809 16:24:21.229562 17982 raft_consensus.cc:374] T e4704d5755fe449693a506af245ba0e9 P 673c507686504b4faccf602508d3b822 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:24:21.229615 17982 raft_consensus.cc:729] T e4704d5755fe449693a506af245ba0e9 P 673c507686504b4faccf602508d3b822 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 673c507686504b4faccf602508d3b822, State: Initialized, Role: FOLLOWER
I20260809 16:24:21.229610 17979 consensus_queue.cc:260] T e4704d5755fe449693a506af245ba0e9 P a96001122a9a40b8a7f65d6e9510177e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a96001122a9a40b8a7f65d6e9510177e" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 42055 } } peers { permanent_uuid: "99915d8b5bc04a8d8ff2e03ec84130a4" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 33953 } } peers { permanent_uuid: "673c507686504b4faccf602508d3b822" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 39649 } }
I20260809 16:24:21.229763 17982 consensus_queue.cc:260] T e4704d5755fe449693a506af245ba0e9 P 673c507686504b4faccf602508d3b822 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a96001122a9a40b8a7f65d6e9510177e" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 42055 } } peers { permanent_uuid: "99915d8b5bc04a8d8ff2e03ec84130a4" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 33953 } } peers { permanent_uuid: "673c507686504b4faccf602508d3b822" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 39649 } }
I20260809 16:24:21.230073 17761 heartbeater.cc:503] Master 127.21.51.190:41521 was elected leader, sending a full tablet report...
I20260809 16:24:21.230077 17982 ts_tablet_manager.cc:1428] T e4704d5755fe449693a506af245ba0e9 P 673c507686504b4faccf602508d3b822: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260809 16:24:21.230239 17560 heartbeater.cc:503] Master 127.21.51.190:41521 was elected leader, sending a full tablet report...
I20260809 16:24:21.230338 17979 ts_tablet_manager.cc:1428] T e4704d5755fe449693a506af245ba0e9 P a96001122a9a40b8a7f65d6e9510177e: Time spent starting tablet: real 0.003s user 0.003s sys 0.001s
I20260809 16:24:21.230520 17986 tablet_bootstrap.cc:492] T e4704d5755fe449693a506af245ba0e9 P 99915d8b5bc04a8d8ff2e03ec84130a4: Bootstrap starting.
I20260809 16:24:21.231190 17986 tablet_bootstrap.cc:654] T e4704d5755fe449693a506af245ba0e9 P 99915d8b5bc04a8d8ff2e03ec84130a4: Neither blocks nor log segments found. Creating new log.
I20260809 16:24:21.231639 17986 log.cc:824] T e4704d5755fe449693a506af245ba0e9 P 99915d8b5bc04a8d8ff2e03ec84130a4: Log is configured to *not* fsync() on all Append() calls
I20260809 16:24:21.232429 17986 tablet_bootstrap.cc:492] T e4704d5755fe449693a506af245ba0e9 P 99915d8b5bc04a8d8ff2e03ec84130a4: No bootstrap required, opened a new log
I20260809 16:24:21.232517 17986 ts_tablet_manager.cc:1397] T e4704d5755fe449693a506af245ba0e9 P 99915d8b5bc04a8d8ff2e03ec84130a4: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 16:24:21.233959 17986 raft_consensus.cc:348] T e4704d5755fe449693a506af245ba0e9 P 99915d8b5bc04a8d8ff2e03ec84130a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a96001122a9a40b8a7f65d6e9510177e" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 42055 } } peers { permanent_uuid: "99915d8b5bc04a8d8ff2e03ec84130a4" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 33953 } } peers { permanent_uuid: "673c507686504b4faccf602508d3b822" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 39649 } }
I20260809 16:24:21.234109 17986 raft_consensus.cc:374] T e4704d5755fe449693a506af245ba0e9 P 99915d8b5bc04a8d8ff2e03ec84130a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:24:21.234157 17986 raft_consensus.cc:729] T e4704d5755fe449693a506af245ba0e9 P 99915d8b5bc04a8d8ff2e03ec84130a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99915d8b5bc04a8d8ff2e03ec84130a4, State: Initialized, Role: FOLLOWER
I20260809 16:24:21.234319 17986 consensus_queue.cc:260] T e4704d5755fe449693a506af245ba0e9 P 99915d8b5bc04a8d8ff2e03ec84130a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a96001122a9a40b8a7f65d6e9510177e" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 42055 } } peers { permanent_uuid: "99915d8b5bc04a8d8ff2e03ec84130a4" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 33953 } } peers { permanent_uuid: "673c507686504b4faccf602508d3b822" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 39649 } }
I20260809 16:24:21.234609 17986 ts_tablet_manager.cc:1428] T e4704d5755fe449693a506af245ba0e9 P 99915d8b5bc04a8d8ff2e03ec84130a4: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260809 16:24:21.234700 17957 heartbeater.cc:503] Master 127.21.51.190:41521 was elected leader, sending a full tablet report...
W20260809 16:24:21.268831 17763 tablet.cc:2369] T e4704d5755fe449693a506af245ba0e9 P a96001122a9a40b8a7f65d6e9510177e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 16:24:21.320622 17990 raft_consensus.cc:482] T e4704d5755fe449693a506af245ba0e9 P 673c507686504b4faccf602508d3b822 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 16:24:21.320772 17990 raft_consensus.cc:504] T e4704d5755fe449693a506af245ba0e9 P 673c507686504b4faccf602508d3b822 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a96001122a9a40b8a7f65d6e9510177e" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 42055 } } peers { permanent_uuid: "99915d8b5bc04a8d8ff2e03ec84130a4" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 33953 } } peers { permanent_uuid: "673c507686504b4faccf602508d3b822" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 39649 } }
I20260809 16:24:21.321153 17990 leader_election.cc:288] T e4704d5755fe449693a506af245ba0e9 P 673c507686504b4faccf602508d3b822 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a96001122a9a40b8a7f65d6e9510177e (127.21.51.130:42055), 99915d8b5bc04a8d8ff2e03ec84130a4 (127.21.51.131:33953)
I20260809 16:24:21.324214 17563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:21.321339 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.130:42055 (local address 127.21.51.129:55259)
0809 16:24:21.321424 (+ 85us) negotiation.cc:107] Waiting for socket to connect
0809 16:24:21.321438 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 16:24:21.321535 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:24:21.321699 (+ 164us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:24:21.321701 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:24:21.321703 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:24:21.321798 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:24:21.321801 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:24:21.322524 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:24:21.322525 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:24:21.323324 (+ 799us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:24:21.323326 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:24:21.323936 (+ 610us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:24:21.323938 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:24:21.324024 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:24:21.324028 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 16:24:21.324080 (+ 52us) client_negotiation.cc:240] Negotiation successful
0809 16:24:21.324109 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":60,"mutex_wait_us":1}
I20260809 16:24:21.324273 17977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:21.321603 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.51.129:52981 (local address 127.21.51.131:33953)
0809 16:24:21.321656 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0809 16:24:21.321659 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 16:24:21.321785 (+ 126us) server_negotiation.cc:407] Connection header received
0809 16:24:21.321815 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:24:21.321816 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:24:21.321826 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:24:21.321879 (+ 53us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:24:21.322046 (+ 167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:24:21.322528 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:24:21.323513 (+ 985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:24:21.323910 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:24:21.323988 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:24:21.323997 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0809 16:24:21.324159 (+ 162us) server_negotiation.cc:299] Negotiation successful
0809 16:24:21.324200 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260809 16:24:21.324272 17975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:21.321436 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.51.129:55259 (local address 127.21.51.130:42055)
0809 16:24:21.321500 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0809 16:24:21.321503 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 16:24:21.321515 (+ 12us) server_negotiation.cc:407] Connection header received
0809 16:24:21.321609 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:24:21.321611 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:24:21.321624 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:24:21.321684 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:24:21.321899 (+ 215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:24:21.322442 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:24:21.323420 (+ 978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:24:21.323825 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:24:21.323879 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:24:21.323886 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0809 16:24:21.324109 (+ 223us) server_negotiation.cc:299] Negotiation successful
0809 16:24:21.324151 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260809 16:24:21.324332 17993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:21.321471 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.131:33953 (local address 127.21.51.129:52981)
0809 16:24:21.321676 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0809 16:24:21.321685 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 16:24:21.321745 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:24:21.321899 (+ 154us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:24:21.321900 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:24:21.321902 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:24:21.321973 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:24:21.321974 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:24:21.322612 (+ 638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:24:21.322613 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:24:21.323421 (+ 808us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:24:21.323424 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:24:21.323999 (+ 575us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:24:21.324000 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:24:21.324055 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:24:21.324059 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 16:24:21.324137 (+ 78us) client_negotiation.cc:240] Negotiation successful
0809 16:24:21.324170 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":125,"threads_started":1}
I20260809 16:24:21.324625 17901 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e4704d5755fe449693a506af245ba0e9" candidate_uuid: "673c507686504b4faccf602508d3b822" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99915d8b5bc04a8d8ff2e03ec84130a4" is_pre_election: true
I20260809 16:24:21.324631 17692 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e4704d5755fe449693a506af245ba0e9" candidate_uuid: "673c507686504b4faccf602508d3b822" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a96001122a9a40b8a7f65d6e9510177e" is_pre_election: true
I20260809 16:24:21.324806 17901 raft_consensus.cc:2454] T e4704d5755fe449693a506af245ba0e9 P 99915d8b5bc04a8d8ff2e03ec84130a4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 673c507686504b4faccf602508d3b822 in term 0.
I20260809 16:24:21.324806 17692 raft_consensus.cc:2454] T e4704d5755fe449693a506af245ba0e9 P a96001122a9a40b8a7f65d6e9510177e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 673c507686504b4faccf602508d3b822 in term 0.
I20260809 16:24:21.325067 17386 leader_election.cc:302] T e4704d5755fe449693a506af245ba0e9 P 673c507686504b4faccf602508d3b822 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 673c507686504b4faccf602508d3b822, a96001122a9a40b8a7f65d6e9510177e; no voters:
I20260809 16:24:21.325201 17990 raft_consensus.cc:2781] T e4704d5755fe449693a506af245ba0e9 P 673c507686504b4faccf602508d3b822 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 16:24:21.325304 17990 raft_consensus.cc:482] T e4704d5755fe449693a506af245ba0e9 P 673c507686504b4faccf602508d3b822 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 16:24:21.325338 17990 raft_consensus.cc:3037] T e4704d5755fe449693a506af245ba0e9 P 673c507686504b4faccf602508d3b822 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:24:21.326102 17990 raft_consensus.cc:504] T e4704d5755fe449693a506af245ba0e9 P 673c507686504b4faccf602508d3b822 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a96001122a9a40b8a7f65d6e9510177e" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 42055 } } peers { permanent_uuid: "99915d8b5bc04a8d8ff2e03ec84130a4" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 33953 } } peers { permanent_uuid: "673c507686504b4faccf602508d3b822" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 39649 } }
I20260809 16:24:21.326246 17990 leader_election.cc:288] T e4704d5755fe449693a506af245ba0e9 P 673c507686504b4faccf602508d3b822 [CANDIDATE]: Term 1 election: Requested vote from peers a96001122a9a40b8a7f65d6e9510177e (127.21.51.130:42055), 99915d8b5bc04a8d8ff2e03ec84130a4 (127.21.51.131:33953)
I20260809 16:24:21.326411 17692 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e4704d5755fe449693a506af245ba0e9" candidate_uuid: "673c507686504b4faccf602508d3b822" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a96001122a9a40b8a7f65d6e9510177e"
I20260809 16:24:21.326443 17901 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e4704d5755fe449693a506af245ba0e9" candidate_uuid: "673c507686504b4faccf602508d3b822" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99915d8b5bc04a8d8ff2e03ec84130a4"
I20260809 16:24:21.326498 17692 raft_consensus.cc:3037] T e4704d5755fe449693a506af245ba0e9 P a96001122a9a40b8a7f65d6e9510177e [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:24:21.326527 17901 raft_consensus.cc:3037] T e4704d5755fe449693a506af245ba0e9 P 99915d8b5bc04a8d8ff2e03ec84130a4 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:24:21.327211 17692 raft_consensus.cc:2454] T e4704d5755fe449693a506af245ba0e9 P a96001122a9a40b8a7f65d6e9510177e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 673c507686504b4faccf602508d3b822 in term 1.
I20260809 16:24:21.327229 17901 raft_consensus.cc:2454] T e4704d5755fe449693a506af245ba0e9 P 99915d8b5bc04a8d8ff2e03ec84130a4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 673c507686504b4faccf602508d3b822 in term 1.
I20260809 16:24:21.327402 17386 leader_election.cc:302] T e4704d5755fe449693a506af245ba0e9 P 673c507686504b4faccf602508d3b822 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 673c507686504b4faccf602508d3b822, a96001122a9a40b8a7f65d6e9510177e; no voters:
I20260809 16:24:21.327519 17990 raft_consensus.cc:2781] T e4704d5755fe449693a506af245ba0e9 P 673c507686504b4faccf602508d3b822 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:24:21.327684 17990 raft_consensus.cc:686] T e4704d5755fe449693a506af245ba0e9 P 673c507686504b4faccf602508d3b822 [term 1 LEADER]: Becoming Leader. State: Replica: 673c507686504b4faccf602508d3b822, State: Running, Role: LEADER
I20260809 16:24:21.327803 17990 consensus_queue.cc:237] T e4704d5755fe449693a506af245ba0e9 P 673c507686504b4faccf602508d3b822 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a96001122a9a40b8a7f65d6e9510177e" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 42055 } } peers { permanent_uuid: "99915d8b5bc04a8d8ff2e03ec84130a4" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 33953 } } peers { permanent_uuid: "673c507686504b4faccf602508d3b822" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 39649 } }
I20260809 16:24:21.328742 17272 catalog_manager.cc:5495] T e4704d5755fe449693a506af245ba0e9 P 673c507686504b4faccf602508d3b822 reported cstate change: term changed from 0 to 1, leader changed from <none> to 673c507686504b4faccf602508d3b822 (127.21.51.129). New cstate: current_term: 1 leader_uuid: "673c507686504b4faccf602508d3b822" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a96001122a9a40b8a7f65d6e9510177e" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 42055 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "99915d8b5bc04a8d8ff2e03ec84130a4" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 33953 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "673c507686504b4faccf602508d3b822" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 39649 } health_report { overall_health: HEALTHY } } }
I20260809 16:24:21.340209 17974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:21.337696 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34410 (local address 127.21.51.129:39649)
0809 16:24:21.337758 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0809 16:24:21.337760 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:24:21.337769 (+ 9us) server_negotiation.cc:407] Connection header received
0809 16:24:21.337900 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:24:21.337901 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:24:21.337919 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:24:21.338008 (+ 89us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 16:24:21.338150 (+ 142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:24:21.338609 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:24:21.339420 (+ 811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:24:21.339625 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:24:21.339698 (+ 73us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:24:21.339873 (+ 175us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 16:24:21.339983 (+ 110us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 16:24:21.340092 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0809 16:24:21.340123 (+ 31us) server_negotiation.cc:299] Negotiation successful
0809 16:24:21.340133 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260809 16:24:21.341996 17901 raft_consensus.cc:1261] T e4704d5755fe449693a506af245ba0e9 P 99915d8b5bc04a8d8ff2e03ec84130a4 [term 1 FOLLOWER]: Refusing update from remote peer 673c507686504b4faccf602508d3b822: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 16:24:21.341996 17692 raft_consensus.cc:1261] T e4704d5755fe449693a506af245ba0e9 P a96001122a9a40b8a7f65d6e9510177e [term 1 FOLLOWER]: Refusing update from remote peer 673c507686504b4faccf602508d3b822: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 16:24:21.342381 17990 consensus_queue.cc:1035] T e4704d5755fe449693a506af245ba0e9 P 673c507686504b4faccf602508d3b822 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a96001122a9a40b8a7f65d6e9510177e" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 42055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 16:24:21.342541 17997 consensus_queue.cc:1035] T e4704d5755fe449693a506af245ba0e9 P 673c507686504b4faccf602508d3b822 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99915d8b5bc04a8d8ff2e03ec84130a4" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 33953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 16:24:21.344919 18002 mvcc.cc:205] Tried to move back new op lower bound from 7316654740854165504 to 7316654740799344640. Current Snapshot: MvccSnapshot[applied={T|T < 7316654740854165504}]
I20260809 16:24:21.345362 18005 mvcc.cc:205] Tried to move back new op lower bound from 7316654740854165504 to 7316654740799344640. Current Snapshot: MvccSnapshot[applied={T|T < 7316654740854165504}]
I20260809 16:24:21.347512 17272 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59538:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 16:24:21.347608 17272 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59538:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 16:24:21.348400 17272 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4963572a4a194f24b98c7656e9ed70a6: Sending DeleteTablet for 3 replicas of tablet e4704d5755fe449693a506af245ba0e9
I20260809 16:24:21.348747 17456 tablet_service.cc:1514] Processing DeleteTablet for tablet e4704d5755fe449693a506af245ba0e9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:24:21 UTC) from {username='kudu', principal='kudu/127.21.51.190@KRBTEST.COM'} at 127.0.0.1:34400
I20260809 16:24:21.348768 17657 tablet_service.cc:1514] Processing DeleteTablet for tablet e4704d5755fe449693a506af245ba0e9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:24:21 UTC) from {username='kudu', principal='kudu/127.21.51.190@KRBTEST.COM'} at 127.0.0.1:36752
I20260809 16:24:21.348965 18018 tablet_replica.cc:330] T e4704d5755fe449693a506af245ba0e9 P 673c507686504b4faccf602508d3b822: stopping tablet replica
I20260809 16:24:21.349000 17860 tablet_service.cc:1514] Processing DeleteTablet for tablet e4704d5755fe449693a506af245ba0e9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:24:21 UTC) from {username='kudu', principal='kudu/127.21.51.190@KRBTEST.COM'} at 127.0.0.1:32926
I20260809 16:24:21.349062 18018 raft_consensus.cc:2229] T e4704d5755fe449693a506af245ba0e9 P 673c507686504b4faccf602508d3b822 [term 1 LEADER]: Raft consensus shutting down.
I20260809 16:24:21.349161 18018 raft_consensus.cc:2258] T e4704d5755fe449693a506af245ba0e9 P 673c507686504b4faccf602508d3b822 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 16:24:21.349277 18021 tablet_replica.cc:330] T e4704d5755fe449693a506af245ba0e9 P 99915d8b5bc04a8d8ff2e03ec84130a4: stopping tablet replica
I20260809 16:24:21.349507 18021 raft_consensus.cc:2229] T e4704d5755fe449693a506af245ba0e9 P 99915d8b5bc04a8d8ff2e03ec84130a4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 16:24:21.349516 21710 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrNqUCn/build/release/bin/kudu with pid 17363
I20260809 16:24:21.352176 18020 tablet_replica.cc:330] T e4704d5755fe449693a506af245ba0e9 P a96001122a9a40b8a7f65d6e9510177e: stopping tablet replica
I20260809 16:24:21.352284 18020 raft_consensus.cc:2229] T e4704d5755fe449693a506af245ba0e9 P a96001122a9a40b8a7f65d6e9510177e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 16:24:21.352424 18020 raft_consensus.cc:2258] T e4704d5755fe449693a506af245ba0e9 P a96001122a9a40b8a7f65d6e9510177e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 16:24:21.359797 18021 raft_consensus.cc:2258] T e4704d5755fe449693a506af245ba0e9 P 99915d8b5bc04a8d8ff2e03ec84130a4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 16:24:21.360280 18021 ts_tablet_manager.cc:1905] T e4704d5755fe449693a506af245ba0e9 P 99915d8b5bc04a8d8ff2e03ec84130a4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 16:24:21.361045 18020 ts_tablet_manager.cc:1905] T e4704d5755fe449693a506af245ba0e9 P a96001122a9a40b8a7f65d6e9510177e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 16:24:21.362082 18021 ts_tablet_manager.cc:1918] T e4704d5755fe449693a506af245ba0e9 P 99915d8b5bc04a8d8ff2e03ec84130a4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 16:24:21.362167 18021 log.cc:1196] T e4704d5755fe449693a506af245ba0e9 P 99915d8b5bc04a8d8ff2e03ec84130a4: Deleting WAL directory at /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786292417722232-21710-0/minicluster-data/ts-2/wal/wals/e4704d5755fe449693a506af245ba0e9
I20260809 16:24:21.362464 18021 ts_tablet_manager.cc:1939] T e4704d5755fe449693a506af245ba0e9 P 99915d8b5bc04a8d8ff2e03ec84130a4: Deleting consensus metadata
I20260809 16:24:21.362757 18020 ts_tablet_manager.cc:1918] T e4704d5755fe449693a506af245ba0e9 P a96001122a9a40b8a7f65d6e9510177e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 16:24:21.362843 18020 log.cc:1196] T e4704d5755fe449693a506af245ba0e9 P a96001122a9a40b8a7f65d6e9510177e: Deleting WAL directory at /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786292417722232-21710-0/minicluster-data/ts-1/wal/wals/e4704d5755fe449693a506af245ba0e9
I20260809 16:24:21.362939 17255 catalog_manager.cc:4841] TS 99915d8b5bc04a8d8ff2e03ec84130a4 (127.21.51.131:33953): tablet e4704d5755fe449693a506af245ba0e9 (table test-table [id=a62afb4bb73949a9971f85f4493e2989]) successfully deleted
I20260809 16:24:21.363126 18020 ts_tablet_manager.cc:1939] T e4704d5755fe449693a506af245ba0e9 P a96001122a9a40b8a7f65d6e9510177e: Deleting consensus metadata
I20260809 16:24:21.363528 17255 catalog_manager.cc:4841] TS a96001122a9a40b8a7f65d6e9510177e (127.21.51.130:42055): tablet e4704d5755fe449693a506af245ba0e9 (table test-table [id=a62afb4bb73949a9971f85f4493e2989]) successfully deleted
W20260809 16:24:21.364403 17254 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.21.51.129:39649 (error 108)
I20260809 16:24:21.365005 17972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:21.364861 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.129:39649 (local address 127.0.0.1:34416)
0809 16:24:21.364918 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0809 16:24:21.364964 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.51.129:39649: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260809 16:24:21.365056 21710 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrNqUCn/build/release/bin/kudu with pid 17566
W20260809 16:24:21.365149 17254 catalog_manager.cc:4584] TS 673c507686504b4faccf602508d3b822 (127.21.51.129:39649): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet e4704d5755fe449693a506af245ba0e9: Network error: Client connection negotiation failed: client connection to 127.21.51.129:39649: connect: Connection refused (error 111)
I20260809 16:24:21.374964 21710 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrNqUCn/build/release/bin/kudu with pid 17770
I20260809 16:24:21.387413 21710 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrNqUCn/build/release/bin/kudu with pid 17228
2026-08-09T16:24:21Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (2236 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-08-09T16:24:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T16:24:21Z Disabled control of system clock
I20260809 16:24:21.426590 21710 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrNqUCn/build/release/bin/kudu
/tmp/dist-test-taskrNqUCn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786292417722232-21710-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786292417722232-21710-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786292417722232-21710-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786292417722232-21710-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.51.190:34253
--webserver_interface=127.21.51.190
--webserver_port=0
--builtin_ntp_servers=127.21.51.148:43669
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.51.190:34253
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 16:24:21.509475 18029 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:24:21.509691 18029 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:24:21.509712 18029 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:24:21.510962 18029 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 16:24:21.511003 18029 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:24:21.511016 18029 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:24:21.511030 18029 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 16:24:21.511042 18029 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 16:24:21.512421 18029 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.51.148:43669
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786292417722232-21710-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786292417722232-21710-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.51.190:34253
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.51.190:34253
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786292417722232-21710-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.51.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18029
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786292417722232-21710-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:24:21.512714 18029 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:24:21.513043 18029 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:24:21.518203 18037 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:24:21.518301 18041 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:24:21.518218 18038 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:24:21.518479 18029 server_base.cc:1034] running on GCE node
I20260809 16:24:21.518776 18029 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:24:21.519074 18029 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:24:21.520277 18029 hybrid_clock.cc:648] HybridClock initialized: now 1786292661520263 us; error 35 us; skew 500 ppm
I20260809 16:24:21.521726 18029 webserver.cc:460] Webserver started at http://127.21.51.190:35191/ using document root <none> and password file <none>
I20260809 16:24:21.522051 18029 fs_manager.cc:359] Metadata directory not provided
I20260809 16:24:21.522107 18029 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:24:21.522249 18029 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:24:21.523391 18029 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786292417722232-21710-0/minicluster-data/master-0/data/instance:
uuid: "6b8a856b37a24e5f80d1a8c25dd724c9"
format_stamp: "Formatted at 2026-08-09 16:24:21 on dist-test-slave-s11t"
server_key: "b9c0945935b303cb4ea659a6a5561ecd"
server_key_iv: "ae95bbffddac7c5b9a4a650fb51c3b2d"
server_key_version: "encryptionkey@0"
I20260809 16:24:21.523756 18029 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786292417722232-21710-0/minicluster-data/master-0/wal/instance:
uuid: "6b8a856b37a24e5f80d1a8c25dd724c9"
format_stamp: "Formatted at 2026-08-09 16:24:21 on dist-test-slave-s11t"
server_key: "b9c0945935b303cb4ea659a6a5561ecd"
server_key_iv: "ae95bbffddac7c5b9a4a650fb51c3b2d"
server_key_version: "encryptionkey@0"
I20260809 16:24:21.525204 18029 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 16:24:21.526053 18049 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:24:21.526301 18029 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 16:24:21.526420 18029 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786292417722232-21710-0/minicluster-data/master-0/data,/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786292417722232-21710-0/minicluster-data/master-0/wal
uuid: "6b8a856b37a24e5f80d1a8c25dd724c9"
format_stamp: "Formatted at 2026-08-09 16:24:21 on dist-test-slave-s11t"
server_key: "b9c0945935b303cb4ea659a6a5561ecd"
server_key_iv: "ae95bbffddac7c5b9a4a650fb51c3b2d"
server_key_version: "encryptionkey@0"
I20260809 16:24:21.526499 18029 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786292417722232-21710-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786292417722232-21710-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786292417722232-21710-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:24:21.534129 18029 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:24:21.534416 18029 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:24:21.534566 18029 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:24:21.538914 18029 rpc_server.cc:307] RPC server started. Bound to: 127.21.51.190:34253
I20260809 16:24:21.539083 18136 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.51.190:34253 every 8 connection(s)
I20260809 16:24:21.539331 18029 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786292417722232-21710-0/minicluster-data/master-0/data/info.pb
I20260809 16:24:21.540293 18138 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:24:21.541172 21710 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrNqUCn/build/release/bin/kudu as pid 18029
I20260809 16:24:21.541314 21710 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786292417722232-21710-0/minicluster-data/master-0/wal/instance
I20260809 16:24:21.541529 21710 external_mini_cluster.cc:1428] Setting key 93eabe731f9929e1648c738c8f7c34e7
I20260809 16:24:21.543094 18138 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6b8a856b37a24e5f80d1a8c25dd724c9: Bootstrap starting.
I20260809 16:24:21.543736 18138 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6b8a856b37a24e5f80d1a8c25dd724c9: Neither blocks nor log segments found. Creating new log.
I20260809 16:24:21.544021 18138 log.cc:824] T 00000000000000000000000000000000 P 6b8a856b37a24e5f80d1a8c25dd724c9: Log is configured to *not* fsync() on all Append() calls
I20260809 16:24:21.544788 18138 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6b8a856b37a24e5f80d1a8c25dd724c9: No bootstrap required, opened a new log
I20260809 16:24:21.546684 18138 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6b8a856b37a24e5f80d1a8c25dd724c9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b8a856b37a24e5f80d1a8c25dd724c9" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 34253 } }
I20260809 16:24:21.546837 18138 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6b8a856b37a24e5f80d1a8c25dd724c9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:24:21.546896 18138 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6b8a856b37a24e5f80d1a8c25dd724c9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6b8a856b37a24e5f80d1a8c25dd724c9, State: Initialized, Role: FOLLOWER
I20260809 16:24:21.547058 18138 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6b8a856b37a24e5f80d1a8c25dd724c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b8a856b37a24e5f80d1a8c25dd724c9" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 34253 } }
I20260809 16:24:21.547147 18138 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6b8a856b37a24e5f80d1a8c25dd724c9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 16:24:21.547194 18138 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6b8a856b37a24e5f80d1a8c25dd724c9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 16:24:21.547256 18138 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6b8a856b37a24e5f80d1a8c25dd724c9 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:24:21.547358 18141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:21.542107 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55928 (local address 127.21.51.190:34253)
0809 16:24:21.542309 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0809 16:24:21.542316 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 16:24:21.542345 (+ 29us) server_negotiation.cc:407] Connection header received
0809 16:24:21.542867 (+ 522us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:24:21.542873 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:24:21.542943 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:24:21.543058 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:24:21.543307 (+ 249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:24:21.544285 (+ 978us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:24:21.545217 (+ 932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:24:21.545654 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:24:21.545792 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:24:21.546543 (+ 751us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:24:21.546554 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:24:21.546554 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 16:24:21.546713 (+ 159us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:24:21.546744 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 16:24:21.546755 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:24:21.546828 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0809 16:24:21.546986 (+ 158us) server_negotiation.cc:299] Negotiation successful
0809 16:24:21.547068 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":125,"thread_start_us":91,"threads_started":1}
I20260809 16:24:21.547861 18138 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6b8a856b37a24e5f80d1a8c25dd724c9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b8a856b37a24e5f80d1a8c25dd724c9" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 34253 } }
I20260809 16:24:21.547966 18138 leader_election.cc:302] T 00000000000000000000000000000000 P 6b8a856b37a24e5f80d1a8c25dd724c9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6b8a856b37a24e5f80d1a8c25dd724c9; no voters:
I20260809 16:24:21.548139 18138 leader_election.cc:288] T 00000000000000000000000000000000 P 6b8a856b37a24e5f80d1a8c25dd724c9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 16:24:21.548182 18144 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6b8a856b37a24e5f80d1a8c25dd724c9 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:24:21.548336 18144 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6b8a856b37a24e5f80d1a8c25dd724c9 [term 1 LEADER]: Becoming Leader. State: Replica: 6b8a856b37a24e5f80d1a8c25dd724c9, State: Running, Role: LEADER
I20260809 16:24:21.548422 18144 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6b8a856b37a24e5f80d1a8c25dd724c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b8a856b37a24e5f80d1a8c25dd724c9" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 34253 } }
I20260809 16:24:21.548429 18138 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6b8a856b37a24e5f80d1a8c25dd724c9 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 16:24:21.548813 18146 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6b8a856b37a24e5f80d1a8c25dd724c9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6b8a856b37a24e5f80d1a8c25dd724c9. Latest consensus state: current_term: 1 leader_uuid: "6b8a856b37a24e5f80d1a8c25dd724c9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b8a856b37a24e5f80d1a8c25dd724c9" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 34253 } } }
I20260809 16:24:21.548820 18145 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6b8a856b37a24e5f80d1a8c25dd724c9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6b8a856b37a24e5f80d1a8c25dd724c9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b8a856b37a24e5f80d1a8c25dd724c9" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 34253 } } }
I20260809 16:24:21.548964 18146 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6b8a856b37a24e5f80d1a8c25dd724c9 [sys.catalog]: This master's current role is: LEADER
I20260809 16:24:21.549014 18145 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6b8a856b37a24e5f80d1a8c25dd724c9 [sys.catalog]: This master's current role is: LEADER
I20260809 16:24:21.549355 18155 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 16:24:21.549883 18155 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 16:24:21.551390 18155 catalog_manager.cc:1347] Generated new cluster ID: 4cd20a6c846f494f95a986add4d3f485
I20260809 16:24:21.551445 18155 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 16:24:21.599813 18155 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 16:24:21.600383 18155 catalog_manager.cc:1504] Loading token signing keys...
I20260809 16:24:21.608007 18155 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6b8a856b37a24e5f80d1a8c25dd724c9: Generated new TSK 0
I20260809 16:24:21.608199 18155 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 16:24:21.618234 21710 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrNqUCn/build/release/bin/kudu
/tmp/dist-test-taskrNqUCn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786292417722232-21710-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786292417722232-21710-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786292417722232-21710-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786292417722232-21710-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.51.129:0
--local_ip_for_outbound_sockets=127.21.51.129
--webserver_interface=127.21.51.129
--webserver_port=0
--tserver_master_addrs=127.21.51.190:34253
--builtin_ntp_servers=127.21.51.148:43669
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 16:24:21.723623 18169 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:24:21.723829 18169 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:24:21.723850 18169 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:24:21.725106 18169 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:24:21.725147 18169 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:24:21.725171 18169 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.51.129
I20260809 16:24:21.726490 18169 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.51.148:43669
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786292417722232-21710-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786292417722232-21710-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.51.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786292417722232-21710-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.51.129
--webserver_port=0
--tserver_master_addrs=127.21.51.190:34253
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18169
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.51.129
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786292417722232-21710-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:24:21.726732 18169 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:24:21.727029 18169 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:24:21.732299 18175 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:24:21.732332 18169 server_base.cc:1034] running on GCE node
W20260809 16:24:21.732396 18181 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:24:21.732488 18176 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:24:21.732821 18169 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:24:21.733119 18169 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:24:21.734262 18169 hybrid_clock.cc:648] HybridClock initialized: now 1786292661734252 us; error 42 us; skew 500 ppm
I20260809 16:24:21.735662 18169 webserver.cc:460] Webserver started at http://127.21.51.129:37465/ using document root <none> and password file <none>
I20260809 16:24:21.735910 18169 fs_manager.cc:359] Metadata directory not provided
I20260809 16:24:21.735984 18169 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:24:21.736146 18169 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:24:21.737262 18169 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786292417722232-21710-0/minicluster-data/ts-0/data/instance:
uuid: "e96c75365fd245a99b23e1d9c2f7d58b"
format_stamp: "Formatted at 2026-08-09 16:24:21 on dist-test-slave-s11t"
server_key: "d43223e4e94b185ec70891a5a8884a18"
server_key_iv: "3614856a99f24575bf4f41c33e774319"
server_key_version: "encryptionkey@0"
I20260809 16:24:21.737632 18169 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786292417722232-21710-0/minicluster-data/ts-0/wal/instance:
uuid: "e96c75365fd245a99b23e1d9c2f7d58b"
format_stamp: "Formatted at 2026-08-09 16:24:21 on dist-test-slave-s11t"
server_key: "d43223e4e94b185ec70891a5a8884a18"
server_key_iv: "3614856a99f24575bf4f41c33e774319"
server_key_version: "encryptionkey@0"
I20260809 16:24:21.738902 18169 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 16:24:21.739607 18187 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:24:21.739790 18169 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 16:24:21.739893 18169 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786292417722232-21710-0/minicluster-data/ts-0/data,/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786292417722232-21710-0/minicluster-data/ts-0/wal
uuid: "e96c75365fd245a99b23e1d9c2f7d58b"
format_stamp: "Formatted at 2026-08-09 16:24:21 on dist-test-slave-s11t"
server_key: "d43223e4e94b185ec70891a5a8884a18"
server_key_iv: "3614856a99f24575bf4f41c33e774319"
server_key_version: "encryptionkey@0"
I20260809 16:24:21.739977 18169 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786292417722232-21710-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786292417722232-21710-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786292417722232-21710-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:24:21.751192 18169 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:24:21.751487 18169 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:24:21.751636 18169 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:24:21.751856 18169 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:24:21.752182 18169 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:24:21.752238 18169 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:24:21.752275 18169 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:24:21.752322 18169 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:24:21.758244 18169 rpc_server.cc:307] RPC server started. Bound to: 127.21.51.129:45995
I20260809 16:24:21.758294 18354 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.51.129:45995 every 8 connection(s)
I20260809 16:24:21.758615 18169 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786292417722232-21710-0/minicluster-data/ts-0/data/info.pb
I20260809 16:24:21.764012 18141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:21.759351 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.51.129:40605 (local address 127.21.51.190:34253)
0809 16:24:21.759412 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0809 16:24:21.759415 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 16:24:21.759646 (+ 231us) server_negotiation.cc:407] Connection header received
0809 16:24:21.760154 (+ 508us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:24:21.760155 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:24:21.760167 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:24:21.760236 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:24:21.761036 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:24:21.761579 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:24:21.762572 (+ 993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:24:21.762792 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:24:21.762859 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:24:21.763679 (+ 820us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:24:21.763680 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:24:21.763681 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 16:24:21.763686 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:24:21.763701 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 16:24:21.763706 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:24:21.763759 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0809 16:24:21.763913 (+ 154us) server_negotiation.cc:299] Negotiation successful
0809 16:24:21.763947 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260809 16:24:21.764241 18357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:21.759290 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:34253 (local address 127.21.51.129:40605)
0809 16:24:21.759483 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0809 16:24:21.759524 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0809 16:24:21.760014 (+ 490us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:24:21.760311 (+ 297us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:24:21.760315 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:24:21.760324 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:24:21.760928 (+ 604us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:24:21.760933 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:24:21.761669 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:24:21.761670 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:24:21.762469 (+ 799us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:24:21.762472 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:24:21.762877 (+ 405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:24:21.762878 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:24:21.762986 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:24:21.763540 (+ 554us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 16:24:21.763541 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:24:21.763569 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 16:24:21.763591 (+ 22us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:24:21.763819 (+ 228us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:24:21.763827 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:24:21.763830 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 16:24:21.763897 (+ 67us) client_negotiation.cc:240] Negotiation successful
0809 16:24:21.763993 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":97,"threads_started":1}
I20260809 16:24:21.764784 18355 heartbeater.cc:347] Connected to a master server at 127.21.51.190:34253
I20260809 16:24:21.764887 18355 heartbeater.cc:464] Registering TS with master...
I20260809 16:24:21.765040 18355 heartbeater.cc:511] Master 127.21.51.190:34253 requested a full tablet report, sending...
I20260809 16:24:21.765208 21710 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrNqUCn/build/release/bin/kudu as pid 18169
I20260809 16:24:21.765329 21710 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786292417722232-21710-0/minicluster-data/ts-0/wal/instance
I20260809 16:24:21.765554 21710 external_mini_cluster.cc:1428] Setting key fe1809cec3613274ed22bb8f82a26032
I20260809 16:24:21.765580 18081 ts_manager.cc:194] Registered new tserver with Master: e96c75365fd245a99b23e1d9c2f7d58b (127.21.51.129:45995)
I20260809 16:24:21.766312 18081 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.21.51.129:40605
I20260809 16:24:21.766878 21710 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrNqUCn/build/release/bin/kudu
/tmp/dist-test-taskrNqUCn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786292417722232-21710-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786292417722232-21710-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786292417722232-21710-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786292417722232-21710-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.51.130:0
--local_ip_for_outbound_sockets=127.21.51.130
--webserver_interface=127.21.51.130
--webserver_port=0
--tserver_master_addrs=127.21.51.190:34253
--builtin_ntp_servers=127.21.51.148:43669
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 16:24:21.855239 18359 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:24:21.855440 18359 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:24:21.855460 18359 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:24:21.856516 18359 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:24:21.856549 18359 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:24:21.856575 18359 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.51.130
I20260809 16:24:21.857859 18359 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.51.148:43669
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786292417722232-21710-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786292417722232-21710-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.51.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786292417722232-21710-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.51.130
--webserver_port=0
--tserver_master_addrs=127.21.51.190:34253
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18359
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.51.130
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786292417722232-21710-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:24:21.858064 18359 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:24:21.858312 18359 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:24:21.863620 18368 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:24:21.863651 18370 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:24:21.863682 18359 server_base.cc:1034] running on GCE node
W20260809 16:24:21.863651 18373 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:24:21.864061 18359 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:24:21.864259 18359 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:24:21.865440 18359 hybrid_clock.cc:648] HybridClock initialized: now 1786292661865427 us; error 29 us; skew 500 ppm
I20260809 16:24:21.866732 18359 webserver.cc:460] Webserver started at http://127.21.51.130:35629/ using document root <none> and password file <none>
I20260809 16:24:21.867004 18359 fs_manager.cc:359] Metadata directory not provided
I20260809 16:24:21.867077 18359 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:24:21.867236 18359 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:24:21.868258 18359 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786292417722232-21710-0/minicluster-data/ts-1/data/instance:
uuid: "d9f5760258c741e7825ba15a5650e3d2"
format_stamp: "Formatted at 2026-08-09 16:24:21 on dist-test-slave-s11t"
server_key: "b21b0b6d4db84c9e54248181d18b8502"
server_key_iv: "0c5dcb8b767bda5594d15ec2b6351475"
server_key_version: "encryptionkey@0"
I20260809 16:24:21.868639 18359 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786292417722232-21710-0/minicluster-data/ts-1/wal/instance:
uuid: "d9f5760258c741e7825ba15a5650e3d2"
format_stamp: "Formatted at 2026-08-09 16:24:21 on dist-test-slave-s11t"
server_key: "b21b0b6d4db84c9e54248181d18b8502"
server_key_iv: "0c5dcb8b767bda5594d15ec2b6351475"
server_key_version: "encryptionkey@0"
I20260809 16:24:21.869988 18359 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260809 16:24:21.870740 18379 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:24:21.870914 18359 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 16:24:21.871017 18359 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786292417722232-21710-0/minicluster-data/ts-1/data,/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786292417722232-21710-0/minicluster-data/ts-1/wal
uuid: "d9f5760258c741e7825ba15a5650e3d2"
format_stamp: "Formatted at 2026-08-09 16:24:21 on dist-test-slave-s11t"
server_key: "b21b0b6d4db84c9e54248181d18b8502"
server_key_iv: "0c5dcb8b767bda5594d15ec2b6351475"
server_key_version: "encryptionkey@0"
I20260809 16:24:21.871095 18359 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786292417722232-21710-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786292417722232-21710-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786292417722232-21710-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:24:21.894294 18359 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:24:21.894611 18359 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:24:21.894771 18359 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:24:21.895009 18359 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:24:21.895349 18359 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:24:21.895412 18359 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:24:21.895447 18359 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:24:21.895475 18359 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:24:21.901237 18359 rpc_server.cc:307] RPC server started. Bound to: 127.21.51.130:45635
I20260809 16:24:21.901288 18550 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.51.130:45635 every 8 connection(s)
I20260809 16:24:21.901636 18359 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786292417722232-21710-0/minicluster-data/ts-1/data/info.pb
I20260809 16:24:21.903004 21710 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrNqUCn/build/release/bin/kudu as pid 18359
I20260809 16:24:21.903111 21710 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786292417722232-21710-0/minicluster-data/ts-1/wal/instance
I20260809 16:24:21.903333 21710 external_mini_cluster.cc:1428] Setting key 98312147679266b47e0eababfba1af28
I20260809 16:24:21.904493 21710 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrNqUCn/build/release/bin/kudu
/tmp/dist-test-taskrNqUCn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786292417722232-21710-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786292417722232-21710-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786292417722232-21710-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786292417722232-21710-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.51.131:0
--local_ip_for_outbound_sockets=127.21.51.131
--webserver_interface=127.21.51.131
--webserver_port=0
--tserver_master_addrs=127.21.51.190:34253
--builtin_ntp_servers=127.21.51.148:43669
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260809 16:24:21.908198 18141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:21.902356 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.51.130:52431 (local address 127.21.51.190:34253)
0809 16:24:21.902400 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0809 16:24:21.902402 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:24:21.902619 (+ 217us) server_negotiation.cc:407] Connection header received
0809 16:24:21.903367 (+ 748us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:24:21.903368 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:24:21.903381 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:24:21.903463 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:24:21.904503 (+ 1040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:24:21.905253 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:24:21.906660 (+ 1407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:24:21.906870 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:24:21.906952 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:24:21.907865 (+ 913us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:24:21.907867 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:24:21.907868 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 16:24:21.907873 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:24:21.907891 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 16:24:21.907897 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:24:21.907958 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0809 16:24:21.908052 (+ 94us) server_negotiation.cc:299] Negotiation successful
0809 16:24:21.908081 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260809 16:24:21.908361 18555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:21.902312 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:34253 (local address 127.21.51.130:52431)
0809 16:24:21.902501 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0809 16:24:21.902534 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0809 16:24:21.903240 (+ 706us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:24:21.903574 (+ 334us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:24:21.903578 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:24:21.903596 (+ 18us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:24:21.904378 (+ 782us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:24:21.904384 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:24:21.905363 (+ 979us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:24:21.905364 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:24:21.906563 (+ 1199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:24:21.906566 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:24:21.906971 (+ 405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:24:21.906972 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:24:21.907115 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:24:21.907711 (+ 596us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 16:24:21.907711 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:24:21.907744 (+ 33us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 16:24:21.907769 (+ 25us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:24:21.907969 (+ 200us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:24:21.907974 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:24:21.907977 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 16:24:21.908036 (+ 59us) client_negotiation.cc:240] Negotiation successful
0809 16:24:21.908126 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":83,"threads_started":1}
I20260809 16:24:21.908977 18552 heartbeater.cc:347] Connected to a master server at 127.21.51.190:34253
I20260809 16:24:21.909076 18552 heartbeater.cc:464] Registering TS with master...
I20260809 16:24:21.909300 18552 heartbeater.cc:511] Master 127.21.51.190:34253 requested a full tablet report, sending...
I20260809 16:24:21.909721 18081 ts_manager.cc:194] Registered new tserver with Master: d9f5760258c741e7825ba15a5650e3d2 (127.21.51.130:45635)
I20260809 16:24:21.910089 18081 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.21.51.130:52431
W20260809 16:24:21.989125 18556 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:24:21.989375 18556 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:24:21.989413 18556 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:24:21.990630 18556 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:24:21.990689 18556 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:24:21.990731 18556 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.51.131
I20260809 16:24:21.992100 18556 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.51.148:43669
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786292417722232-21710-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786292417722232-21710-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.51.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786292417722232-21710-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.51.131
--webserver_port=0
--tserver_master_addrs=127.21.51.190:34253
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18556
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.51.131
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786292417722232-21710-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:24:21.992376 18556 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:24:21.992733 18556 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:24:21.997819 18562 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:24:21.997828 18565 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:24:21.997828 18563 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:24:21.998231 18556 server_base.cc:1034] running on GCE node
I20260809 16:24:21.998441 18556 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:24:21.998641 18556 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:24:21.999850 18556 hybrid_clock.cc:648] HybridClock initialized: now 1786292661999832 us; error 26 us; skew 500 ppm
I20260809 16:24:22.001384 18556 webserver.cc:460] Webserver started at http://127.21.51.131:45969/ using document root <none> and password file <none>
I20260809 16:24:22.001670 18556 fs_manager.cc:359] Metadata directory not provided
I20260809 16:24:22.001785 18556 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:24:22.001961 18556 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:24:22.003244 18556 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786292417722232-21710-0/minicluster-data/ts-2/data/instance:
uuid: "7d2b62849605417c8faf84efe9ffdfcb"
format_stamp: "Formatted at 2026-08-09 16:24:21 on dist-test-slave-s11t"
server_key: "0222afe7e4fb7fffb999b15cdca54493"
server_key_iv: "11d32a03c7be44ecc23244596568c708"
server_key_version: "encryptionkey@0"
I20260809 16:24:22.003639 18556 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786292417722232-21710-0/minicluster-data/ts-2/wal/instance:
uuid: "7d2b62849605417c8faf84efe9ffdfcb"
format_stamp: "Formatted at 2026-08-09 16:24:21 on dist-test-slave-s11t"
server_key: "0222afe7e4fb7fffb999b15cdca54493"
server_key_iv: "11d32a03c7be44ecc23244596568c708"
server_key_version: "encryptionkey@0"
I20260809 16:24:22.005213 18556 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.004s
I20260809 16:24:22.006039 18575 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:24:22.006237 18556 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 16:24:22.006356 18556 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786292417722232-21710-0/minicluster-data/ts-2/data,/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786292417722232-21710-0/minicluster-data/ts-2/wal
uuid: "7d2b62849605417c8faf84efe9ffdfcb"
format_stamp: "Formatted at 2026-08-09 16:24:21 on dist-test-slave-s11t"
server_key: "0222afe7e4fb7fffb999b15cdca54493"
server_key_iv: "11d32a03c7be44ecc23244596568c708"
server_key_version: "encryptionkey@0"
I20260809 16:24:22.006438 18556 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786292417722232-21710-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786292417722232-21710-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786292417722232-21710-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:24:22.042486 18556 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:24:22.042825 18556 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:24:22.043049 18556 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:24:22.043342 18556 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:24:22.043699 18556 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:24:22.043757 18556 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:24:22.043802 18556 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:24:22.043829 18556 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:24:22.050158 18556 rpc_server.cc:307] RPC server started. Bound to: 127.21.51.131:39375
I20260809 16:24:22.050213 18736 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.51.131:39375 every 8 connection(s)
I20260809 16:24:22.050590 18556 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786292417722232-21710-0/minicluster-data/ts-2/data/info.pb
I20260809 16:24:22.053321 21710 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrNqUCn/build/release/bin/kudu as pid 18556
I20260809 16:24:22.053417 21710 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786292417722232-21710-0/minicluster-data/ts-2/wal/instance
I20260809 16:24:22.053639 21710 external_mini_cluster.cc:1428] Setting key 280885cdced155d593b39b76f68f6eb9
I20260809 16:24:22.056181 18141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:22.051313 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.51.131:36345 (local address 127.21.51.190:34253)
0809 16:24:22.051383 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0809 16:24:22.051385 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:24:22.051623 (+ 238us) server_negotiation.cc:407] Connection header received
0809 16:24:22.052093 (+ 470us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:24:22.052094 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:24:22.052104 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:24:22.052155 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:24:22.052875 (+ 720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:24:22.053418 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:24:22.054750 (+ 1332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:24:22.054953 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:24:22.055037 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:24:22.055896 (+ 859us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:24:22.055897 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:24:22.055898 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 16:24:22.055902 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:24:22.055924 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 16:24:22.055928 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:24:22.055998 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0809 16:24:22.056079 (+ 81us) server_negotiation.cc:299] Negotiation successful
0809 16:24:22.056105 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260809 16:24:22.056403 18741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:22.051332 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:34253 (local address 127.21.51.131:36345)
0809 16:24:22.051503 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0809 16:24:22.051536 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0809 16:24:22.052011 (+ 475us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:24:22.052207 (+ 196us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:24:22.052210 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:24:22.052218 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:24:22.052787 (+ 569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:24:22.052795 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:24:22.053545 (+ 750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:24:22.053546 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:24:22.054654 (+ 1108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:24:22.054662 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:24:22.055045 (+ 383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:24:22.055046 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:24:22.055187 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:24:22.055739 (+ 552us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 16:24:22.055739 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:24:22.055767 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 16:24:22.055791 (+ 24us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:24:22.056000 (+ 209us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:24:22.056007 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:24:22.056011 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 16:24:22.056074 (+ 63us) client_negotiation.cc:240] Negotiation successful
0809 16:24:22.056177 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":86,"threads_started":1}
I20260809 16:24:22.056874 18737 heartbeater.cc:347] Connected to a master server at 127.21.51.190:34253
I20260809 16:24:22.056959 18737 heartbeater.cc:464] Registering TS with master...
I20260809 16:24:22.057096 18737 heartbeater.cc:511] Master 127.21.51.190:34253 requested a full tablet report, sending...
I20260809 16:24:22.057443 18081 ts_manager.cc:194] Registered new tserver with Master: 7d2b62849605417c8faf84efe9ffdfcb (127.21.51.131:39375)
I20260809 16:24:22.057806 18081 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.21.51.131:36345
I20260809 16:24:22.065441 21710 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 16:24:22.069774 18141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:22.066789 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55942 (local address 127.21.51.190:34253)
0809 16:24:22.066852 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0809 16:24:22.066854 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:24:22.066997 (+ 143us) server_negotiation.cc:407] Connection header received
0809 16:24:22.067071 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:24:22.067072 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:24:22.067083 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:24:22.067127 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:24:22.067298 (+ 171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:24:22.067758 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:24:22.068499 (+ 741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:24:22.068773 (+ 274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:24:22.068853 (+ 80us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:24:22.069475 (+ 622us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:24:22.069477 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:24:22.069477 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 16:24:22.069482 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:24:22.069495 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 16:24:22.069499 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:24:22.069613 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0809 16:24:22.069685 (+ 72us) server_negotiation.cc:299] Negotiation successful
0809 16:24:22.069695 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260809 16:24:22.071257 18081 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:55942:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 16:24:22.072814 18081 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 16:24:22.079305 18760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:22.075239 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.129:45995 (local address 127.0.0.1:59844)
0809 16:24:22.075555 (+ 316us) negotiation.cc:107] Waiting for socket to connect
0809 16:24:22.075563 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0809 16:24:22.075605 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:24:22.075773 (+ 168us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:24:22.075776 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:24:22.075778 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:24:22.075854 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:24:22.075856 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:24:22.076979 (+ 1123us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:24:22.076980 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:24:22.078227 (+ 1247us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:24:22.078230 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:24:22.079010 (+ 780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:24:22.079011 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:24:22.079090 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:24:22.079102 (+ 12us) client_negotiation.cc:769] Sending connection context
0809 16:24:22.079178 (+ 76us) client_negotiation.cc:240] Negotiation successful
0809 16:24:22.079233 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":298,"mutex_wait_us":3,"thread_start_us":53,"threads_started":1}
I20260809 16:24:22.079316 18766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:22.075371 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59844 (local address 127.21.51.129:45995)
0809 16:24:22.075558 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0809 16:24:22.075562 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 16:24:22.075599 (+ 37us) server_negotiation.cc:407] Connection header received
0809 16:24:22.075655 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:24:22.075656 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:24:22.075668 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:24:22.075768 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:24:22.075945 (+ 177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:24:22.076894 (+ 949us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:24:22.078322 (+ 1428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:24:22.078918 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:24:22.079010 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:24:22.079104 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0809 16:24:22.079194 (+ 90us) server_negotiation.cc:299] Negotiation successful
0809 16:24:22.079233 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":150,"thread_start_us":97,"threads_started":1}
I20260809 16:24:22.079631 18756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:22.075038 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.131:39375 (local address 127.0.0.1:53762)
0809 16:24:22.075203 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0809 16:24:22.075240 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0809 16:24:22.075301 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:24:22.076201 (+ 900us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:24:22.076202 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:24:22.076205 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:24:22.076254 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:24:22.076256 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:24:22.077490 (+ 1234us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:24:22.077492 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:24:22.078456 (+ 964us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:24:22.078459 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:24:22.079371 (+ 912us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:24:22.079372 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:24:22.079468 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:24:22.079473 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 16:24:22.079522 (+ 49us) client_negotiation.cc:240] Negotiation successful
0809 16:24:22.079562 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":61,"threads_started":1}
I20260809 16:24:22.079845 18764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:22.075039 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.130:45635 (local address 127.0.0.1:45260)
0809 16:24:22.075345 (+ 306us) negotiation.cc:107] Waiting for socket to connect
0809 16:24:22.075357 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 16:24:22.075424 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:24:22.075883 (+ 459us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:24:22.075884 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:24:22.075886 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:24:22.075949 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:24:22.075951 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:24:22.077270 (+ 1319us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:24:22.077270 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:24:22.078322 (+ 1052us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:24:22.078325 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:24:22.079315 (+ 990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:24:22.079316 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:24:22.079674 (+ 358us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:24:22.079680 (+ 6us) client_negotiation.cc:769] Sending connection context
0809 16:24:22.079738 (+ 58us) client_negotiation.cc:240] Negotiation successful
0809 16:24:22.079782 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":291,"mutex_wait_us":2,"spinlock_wait_cycles":768,"thread_start_us":69,"threads_started":1}
I20260809 16:24:22.079931 18761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:22.075108 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53762 (local address 127.21.51.131:39375)
0809 16:24:22.076007 (+ 899us) server_negotiation.cc:206] Beginning negotiation
0809 16:24:22.076011 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 16:24:22.076020 (+ 9us) server_negotiation.cc:407] Connection header received
0809 16:24:22.076053 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:24:22.076055 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:24:22.076069 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:24:22.076142 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:24:22.076344 (+ 202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:24:22.077402 (+ 1058us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:24:22.078545 (+ 1143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:24:22.079211 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:24:22.079270 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:24:22.079339 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0809 16:24:22.079798 (+ 459us) server_negotiation.cc:299] Negotiation successful
0809 16:24:22.079837 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":859,"thread_start_us":84,"threads_started":1}
I20260809 16:24:22.080132 18763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:22.075109 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45260 (local address 127.21.51.130:45635)
0809 16:24:22.075703 (+ 594us) server_negotiation.cc:206] Beginning negotiation
0809 16:24:22.075707 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 16:24:22.075735 (+ 28us) server_negotiation.cc:407] Connection header received
0809 16:24:22.075774 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:24:22.075776 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:24:22.075791 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:24:22.075871 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:24:22.076047 (+ 176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:24:22.077180 (+ 1133us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:24:22.078386 (+ 1206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:24:22.079116 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:24:22.079190 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:24:22.079277 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0809 16:24:22.080010 (+ 733us) server_negotiation.cc:299] Negotiation successful
0809 16:24:22.080054 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":562,"thread_start_us":96,"threads_started":1}
I20260809 16:24:22.080070 18270 tablet_service.cc:1467] Processing CreateTablet for tablet 5f2e44295ade4a2490f1234f5469bf04 (DEFAULT_TABLE table=test-table [id=afcea12d64974f8088bc907df4c2adac]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:24:22.080488 18270 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5f2e44295ade4a2490f1234f5469bf04. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:24:22.080533 18641 tablet_service.cc:1467] Processing CreateTablet for tablet 5f2e44295ade4a2490f1234f5469bf04 (DEFAULT_TABLE table=test-table [id=afcea12d64974f8088bc907df4c2adac]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:24:22.080638 18454 tablet_service.cc:1467] Processing CreateTablet for tablet 5f2e44295ade4a2490f1234f5469bf04 (DEFAULT_TABLE table=test-table [id=afcea12d64974f8088bc907df4c2adac]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:24:22.080919 18641 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5f2e44295ade4a2490f1234f5469bf04. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:24:22.080986 18454 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5f2e44295ade4a2490f1234f5469bf04. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:24:22.083479 18767 tablet_bootstrap.cc:492] T 5f2e44295ade4a2490f1234f5469bf04 P e96c75365fd245a99b23e1d9c2f7d58b: Bootstrap starting.
I20260809 16:24:22.084141 18767 tablet_bootstrap.cc:654] T 5f2e44295ade4a2490f1234f5469bf04 P e96c75365fd245a99b23e1d9c2f7d58b: Neither blocks nor log segments found. Creating new log.
I20260809 16:24:22.084157 18769 tablet_bootstrap.cc:492] T 5f2e44295ade4a2490f1234f5469bf04 P 7d2b62849605417c8faf84efe9ffdfcb: Bootstrap starting.
I20260809 16:24:22.084386 18770 tablet_bootstrap.cc:492] T 5f2e44295ade4a2490f1234f5469bf04 P d9f5760258c741e7825ba15a5650e3d2: Bootstrap starting.
I20260809 16:24:22.084487 18767 log.cc:824] T 5f2e44295ade4a2490f1234f5469bf04 P e96c75365fd245a99b23e1d9c2f7d58b: Log is configured to *not* fsync() on all Append() calls
I20260809 16:24:22.084909 18769 tablet_bootstrap.cc:654] T 5f2e44295ade4a2490f1234f5469bf04 P 7d2b62849605417c8faf84efe9ffdfcb: Neither blocks nor log segments found. Creating new log.
I20260809 16:24:22.085116 18770 tablet_bootstrap.cc:654] T 5f2e44295ade4a2490f1234f5469bf04 P d9f5760258c741e7825ba15a5650e3d2: Neither blocks nor log segments found. Creating new log.
I20260809 16:24:22.085240 18769 log.cc:824] T 5f2e44295ade4a2490f1234f5469bf04 P 7d2b62849605417c8faf84efe9ffdfcb: Log is configured to *not* fsync() on all Append() calls
I20260809 16:24:22.085435 18767 tablet_bootstrap.cc:492] T 5f2e44295ade4a2490f1234f5469bf04 P e96c75365fd245a99b23e1d9c2f7d58b: No bootstrap required, opened a new log
I20260809 16:24:22.085443 18770 log.cc:824] T 5f2e44295ade4a2490f1234f5469bf04 P d9f5760258c741e7825ba15a5650e3d2: Log is configured to *not* fsync() on all Append() calls
I20260809 16:24:22.085506 18767 ts_tablet_manager.cc:1397] T 5f2e44295ade4a2490f1234f5469bf04 P e96c75365fd245a99b23e1d9c2f7d58b: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260809 16:24:22.086103 18769 tablet_bootstrap.cc:492] T 5f2e44295ade4a2490f1234f5469bf04 P 7d2b62849605417c8faf84efe9ffdfcb: No bootstrap required, opened a new log
I20260809 16:24:22.086128 18770 tablet_bootstrap.cc:492] T 5f2e44295ade4a2490f1234f5469bf04 P d9f5760258c741e7825ba15a5650e3d2: No bootstrap required, opened a new log
I20260809 16:24:22.086185 18769 ts_tablet_manager.cc:1397] T 5f2e44295ade4a2490f1234f5469bf04 P 7d2b62849605417c8faf84efe9ffdfcb: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 16:24:22.086205 18770 ts_tablet_manager.cc:1397] T 5f2e44295ade4a2490f1234f5469bf04 P d9f5760258c741e7825ba15a5650e3d2: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260809 16:24:22.088076 18770 raft_consensus.cc:348] T 5f2e44295ade4a2490f1234f5469bf04 P d9f5760258c741e7825ba15a5650e3d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9f5760258c741e7825ba15a5650e3d2" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 45635 } } peers { permanent_uuid: "7d2b62849605417c8faf84efe9ffdfcb" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 39375 } } peers { permanent_uuid: "e96c75365fd245a99b23e1d9c2f7d58b" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 45995 } }
I20260809 16:24:22.088236 18770 raft_consensus.cc:374] T 5f2e44295ade4a2490f1234f5469bf04 P d9f5760258c741e7825ba15a5650e3d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:24:22.088163 18767 raft_consensus.cc:348] T 5f2e44295ade4a2490f1234f5469bf04 P e96c75365fd245a99b23e1d9c2f7d58b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9f5760258c741e7825ba15a5650e3d2" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 45635 } } peers { permanent_uuid: "7d2b62849605417c8faf84efe9ffdfcb" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 39375 } } peers { permanent_uuid: "e96c75365fd245a99b23e1d9c2f7d58b" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 45995 } }
I20260809 16:24:22.088281 18770 raft_consensus.cc:729] T 5f2e44295ade4a2490f1234f5469bf04 P d9f5760258c741e7825ba15a5650e3d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d9f5760258c741e7825ba15a5650e3d2, State: Initialized, Role: FOLLOWER
I20260809 16:24:22.088315 18767 raft_consensus.cc:374] T 5f2e44295ade4a2490f1234f5469bf04 P e96c75365fd245a99b23e1d9c2f7d58b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:24:22.088359 18767 raft_consensus.cc:729] T 5f2e44295ade4a2490f1234f5469bf04 P e96c75365fd245a99b23e1d9c2f7d58b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e96c75365fd245a99b23e1d9c2f7d58b, State: Initialized, Role: FOLLOWER
I20260809 16:24:22.088392 18770 consensus_queue.cc:260] T 5f2e44295ade4a2490f1234f5469bf04 P d9f5760258c741e7825ba15a5650e3d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9f5760258c741e7825ba15a5650e3d2" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 45635 } } peers { permanent_uuid: "7d2b62849605417c8faf84efe9ffdfcb" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 39375 } } peers { permanent_uuid: "e96c75365fd245a99b23e1d9c2f7d58b" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 45995 } }
I20260809 16:24:22.088488 18767 consensus_queue.cc:260] T 5f2e44295ade4a2490f1234f5469bf04 P e96c75365fd245a99b23e1d9c2f7d58b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9f5760258c741e7825ba15a5650e3d2" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 45635 } } peers { permanent_uuid: "7d2b62849605417c8faf84efe9ffdfcb" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 39375 } } peers { permanent_uuid: "e96c75365fd245a99b23e1d9c2f7d58b" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 45995 } }
I20260809 16:24:22.088698 18770 ts_tablet_manager.cc:1428] T 5f2e44295ade4a2490f1234f5469bf04 P d9f5760258c741e7825ba15a5650e3d2: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260809 16:24:22.088745 18552 heartbeater.cc:503] Master 127.21.51.190:34253 was elected leader, sending a full tablet report...
I20260809 16:24:22.088768 18767 ts_tablet_manager.cc:1428] T 5f2e44295ade4a2490f1234f5469bf04 P e96c75365fd245a99b23e1d9c2f7d58b: Time spent starting tablet: real 0.003s user 0.000s sys 0.004s
I20260809 16:24:22.088644 18769 raft_consensus.cc:348] T 5f2e44295ade4a2490f1234f5469bf04 P 7d2b62849605417c8faf84efe9ffdfcb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9f5760258c741e7825ba15a5650e3d2" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 45635 } } peers { permanent_uuid: "7d2b62849605417c8faf84efe9ffdfcb" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 39375 } } peers { permanent_uuid: "e96c75365fd245a99b23e1d9c2f7d58b" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 45995 } }
I20260809 16:24:22.088809 18769 raft_consensus.cc:374] T 5f2e44295ade4a2490f1234f5469bf04 P 7d2b62849605417c8faf84efe9ffdfcb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:24:22.088840 18769 raft_consensus.cc:729] T 5f2e44295ade4a2490f1234f5469bf04 P 7d2b62849605417c8faf84efe9ffdfcb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7d2b62849605417c8faf84efe9ffdfcb, State: Initialized, Role: FOLLOWER
I20260809 16:24:22.088914 18355 heartbeater.cc:503] Master 127.21.51.190:34253 was elected leader, sending a full tablet report...
I20260809 16:24:22.089104 18769 consensus_queue.cc:260] T 5f2e44295ade4a2490f1234f5469bf04 P 7d2b62849605417c8faf84efe9ffdfcb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9f5760258c741e7825ba15a5650e3d2" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 45635 } } peers { permanent_uuid: "7d2b62849605417c8faf84efe9ffdfcb" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 39375 } } peers { permanent_uuid: "e96c75365fd245a99b23e1d9c2f7d58b" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 45995 } }
I20260809 16:24:22.089395 18769 ts_tablet_manager.cc:1428] T 5f2e44295ade4a2490f1234f5469bf04 P 7d2b62849605417c8faf84efe9ffdfcb: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 16:24:22.089469 18737 heartbeater.cc:503] Master 127.21.51.190:34253 was elected leader, sending a full tablet report...
W20260809 16:24:22.152209 18554 tablet.cc:2369] T 5f2e44295ade4a2490f1234f5469bf04 P d9f5760258c741e7825ba15a5650e3d2: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 16:24:22.259271 18356 tablet.cc:2369] T 5f2e44295ade4a2490f1234f5469bf04 P e96c75365fd245a99b23e1d9c2f7d58b: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 16:24:22.301189 18740 tablet.cc:2369] T 5f2e44295ade4a2490f1234f5469bf04 P 7d2b62849605417c8faf84efe9ffdfcb: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 16:24:22.389341 18775 raft_consensus.cc:482] T 5f2e44295ade4a2490f1234f5469bf04 P d9f5760258c741e7825ba15a5650e3d2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 16:24:22.389484 18775 raft_consensus.cc:504] T 5f2e44295ade4a2490f1234f5469bf04 P d9f5760258c741e7825ba15a5650e3d2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9f5760258c741e7825ba15a5650e3d2" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 45635 } } peers { permanent_uuid: "7d2b62849605417c8faf84efe9ffdfcb" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 39375 } } peers { permanent_uuid: "e96c75365fd245a99b23e1d9c2f7d58b" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 45995 } }
I20260809 16:24:22.389799 18775 leader_election.cc:288] T 5f2e44295ade4a2490f1234f5469bf04 P d9f5760258c741e7825ba15a5650e3d2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7d2b62849605417c8faf84efe9ffdfcb (127.21.51.131:39375), e96c75365fd245a99b23e1d9c2f7d58b (127.21.51.129:45995)
I20260809 16:24:22.392602 18761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:22.390110 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.51.130:60863 (local address 127.21.51.131:39375)
0809 16:24:22.390161 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0809 16:24:22.390163 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:24:22.390174 (+ 11us) server_negotiation.cc:407] Connection header received
0809 16:24:22.390207 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:24:22.390208 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:24:22.390217 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:24:22.390293 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:24:22.390459 (+ 166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:24:22.390941 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:24:22.391878 (+ 937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:24:22.392245 (+ 367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:24:22.392332 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:24:22.392341 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0809 16:24:22.392492 (+ 151us) server_negotiation.cc:299] Negotiation successful
0809 16:24:22.392531 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260809 16:24:22.392602 18555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:22.390015 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.131:39375 (local address 127.21.51.130:60863)
0809 16:24:22.390078 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0809 16:24:22.390091 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 16:24:22.390138 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:24:22.390298 (+ 160us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:24:22.390299 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:24:22.390301 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:24:22.390382 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:24:22.390384 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:24:22.391030 (+ 646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:24:22.391031 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:24:22.391781 (+ 750us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:24:22.391783 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:24:22.392332 (+ 549us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:24:22.392333 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:24:22.392416 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:24:22.392419 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 16:24:22.392486 (+ 67us) client_negotiation.cc:240] Negotiation successful
0809 16:24:22.392531 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":44}
I20260809 16:24:22.392782 18779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:22.390015 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.129:45995 (local address 127.21.51.130:37221)
0809 16:24:22.390185 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0809 16:24:22.390196 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 16:24:22.390300 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:24:22.390466 (+ 166us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:24:22.390467 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:24:22.390468 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:24:22.390561 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:24:22.390562 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:24:22.391183 (+ 621us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:24:22.391184 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:24:22.391928 (+ 744us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:24:22.391930 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:24:22.392468 (+ 538us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:24:22.392469 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:24:22.392560 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:24:22.392563 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 16:24:22.392631 (+ 68us) client_negotiation.cc:240] Negotiation successful
0809 16:24:22.392669 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":154,"mutex_wait_us":24,"thread_start_us":62,"threads_started":1}
I20260809 16:24:22.392992 18766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:22.390104 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.51.130:37221 (local address 127.21.51.129:45995)
0809 16:24:22.390150 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0809 16:24:22.390153 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 16:24:22.390285 (+ 132us) server_negotiation.cc:407] Connection header received
0809 16:24:22.390359 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:24:22.390360 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:24:22.390394 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:24:22.390463 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:24:22.390636 (+ 173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:24:22.391097 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:24:22.392025 (+ 928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:24:22.392382 (+ 357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:24:22.392467 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:24:22.392474 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0809 16:24:22.392637 (+ 163us) server_negotiation.cc:299] Negotiation successful
0809 16:24:22.392673 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260809 16:24:22.392983 18671 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5f2e44295ade4a2490f1234f5469bf04" candidate_uuid: "d9f5760258c741e7825ba15a5650e3d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7d2b62849605417c8faf84efe9ffdfcb" is_pre_election: true
I20260809 16:24:22.393144 18671 raft_consensus.cc:2454] T 5f2e44295ade4a2490f1234f5469bf04 P 7d2b62849605417c8faf84efe9ffdfcb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9f5760258c741e7825ba15a5650e3d2 in term 0.
I20260809 16:24:22.393276 18300 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5f2e44295ade4a2490f1234f5469bf04" candidate_uuid: "d9f5760258c741e7825ba15a5650e3d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e96c75365fd245a99b23e1d9c2f7d58b" is_pre_election: true
I20260809 16:24:22.393399 18389 leader_election.cc:302] T 5f2e44295ade4a2490f1234f5469bf04 P d9f5760258c741e7825ba15a5650e3d2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d2b62849605417c8faf84efe9ffdfcb, d9f5760258c741e7825ba15a5650e3d2; no voters:
I20260809 16:24:22.393441 18300 raft_consensus.cc:2454] T 5f2e44295ade4a2490f1234f5469bf04 P e96c75365fd245a99b23e1d9c2f7d58b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9f5760258c741e7825ba15a5650e3d2 in term 0.
I20260809 16:24:22.393566 18775 raft_consensus.cc:2781] T 5f2e44295ade4a2490f1234f5469bf04 P d9f5760258c741e7825ba15a5650e3d2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 16:24:22.393630 18775 raft_consensus.cc:482] T 5f2e44295ade4a2490f1234f5469bf04 P d9f5760258c741e7825ba15a5650e3d2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 16:24:22.393667 18775 raft_consensus.cc:3037] T 5f2e44295ade4a2490f1234f5469bf04 P d9f5760258c741e7825ba15a5650e3d2 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:24:22.394366 18775 raft_consensus.cc:504] T 5f2e44295ade4a2490f1234f5469bf04 P d9f5760258c741e7825ba15a5650e3d2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9f5760258c741e7825ba15a5650e3d2" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 45635 } } peers { permanent_uuid: "7d2b62849605417c8faf84efe9ffdfcb" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 39375 } } peers { permanent_uuid: "e96c75365fd245a99b23e1d9c2f7d58b" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 45995 } }
I20260809 16:24:22.394523 18775 leader_election.cc:288] T 5f2e44295ade4a2490f1234f5469bf04 P d9f5760258c741e7825ba15a5650e3d2 [CANDIDATE]: Term 1 election: Requested vote from peers 7d2b62849605417c8faf84efe9ffdfcb (127.21.51.131:39375), e96c75365fd245a99b23e1d9c2f7d58b (127.21.51.129:45995)
I20260809 16:24:22.394654 18671 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5f2e44295ade4a2490f1234f5469bf04" candidate_uuid: "d9f5760258c741e7825ba15a5650e3d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7d2b62849605417c8faf84efe9ffdfcb"
I20260809 16:24:22.394726 18300 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5f2e44295ade4a2490f1234f5469bf04" candidate_uuid: "d9f5760258c741e7825ba15a5650e3d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e96c75365fd245a99b23e1d9c2f7d58b"
I20260809 16:24:22.394764 18671 raft_consensus.cc:3037] T 5f2e44295ade4a2490f1234f5469bf04 P 7d2b62849605417c8faf84efe9ffdfcb [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:24:22.394790 18300 raft_consensus.cc:3037] T 5f2e44295ade4a2490f1234f5469bf04 P e96c75365fd245a99b23e1d9c2f7d58b [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:24:22.395694 18300 raft_consensus.cc:2454] T 5f2e44295ade4a2490f1234f5469bf04 P e96c75365fd245a99b23e1d9c2f7d58b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9f5760258c741e7825ba15a5650e3d2 in term 1.
I20260809 16:24:22.395694 18671 raft_consensus.cc:2454] T 5f2e44295ade4a2490f1234f5469bf04 P 7d2b62849605417c8faf84efe9ffdfcb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9f5760258c741e7825ba15a5650e3d2 in term 1.
I20260809 16:24:22.395864 18389 leader_election.cc:302] T 5f2e44295ade4a2490f1234f5469bf04 P d9f5760258c741e7825ba15a5650e3d2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d2b62849605417c8faf84efe9ffdfcb, d9f5760258c741e7825ba15a5650e3d2; no voters:
I20260809 16:24:22.395967 18775 raft_consensus.cc:2781] T 5f2e44295ade4a2490f1234f5469bf04 P d9f5760258c741e7825ba15a5650e3d2 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:24:22.396157 18775 raft_consensus.cc:686] T 5f2e44295ade4a2490f1234f5469bf04 P d9f5760258c741e7825ba15a5650e3d2 [term 1 LEADER]: Becoming Leader. State: Replica: d9f5760258c741e7825ba15a5650e3d2, State: Running, Role: LEADER
I20260809 16:24:22.396256 18775 consensus_queue.cc:237] T 5f2e44295ade4a2490f1234f5469bf04 P d9f5760258c741e7825ba15a5650e3d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9f5760258c741e7825ba15a5650e3d2" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 45635 } } peers { permanent_uuid: "7d2b62849605417c8faf84efe9ffdfcb" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 39375 } } peers { permanent_uuid: "e96c75365fd245a99b23e1d9c2f7d58b" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 45995 } }
I20260809 16:24:22.397151 18079 catalog_manager.cc:5495] T 5f2e44295ade4a2490f1234f5469bf04 P d9f5760258c741e7825ba15a5650e3d2 reported cstate change: term changed from 0 to 1, leader changed from <none> to d9f5760258c741e7825ba15a5650e3d2 (127.21.51.130). New cstate: current_term: 1 leader_uuid: "d9f5760258c741e7825ba15a5650e3d2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9f5760258c741e7825ba15a5650e3d2" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 45635 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7d2b62849605417c8faf84efe9ffdfcb" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 39375 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e96c75365fd245a99b23e1d9c2f7d58b" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 45995 } health_report { overall_health: UNKNOWN } } }
I20260809 16:24:22.431078 18763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:22.428590 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45268 (local address 127.21.51.130:45635)
0809 16:24:22.428646 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0809 16:24:22.428648 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:24:22.428656 (+ 8us) server_negotiation.cc:407] Connection header received
0809 16:24:22.428758 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:24:22.428759 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:24:22.428786 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:24:22.428850 (+ 64us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 16:24:22.429044 (+ 194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:24:22.429510 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:24:22.430336 (+ 826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:24:22.430521 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:24:22.430573 (+ 52us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:24:22.430700 (+ 127us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 16:24:22.430778 (+ 78us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 16:24:22.430843 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0809 16:24:22.430950 (+ 107us) server_negotiation.cc:299] Negotiation successful
0809 16:24:22.430962 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260809 16:24:22.433763 18300 raft_consensus.cc:1261] T 5f2e44295ade4a2490f1234f5469bf04 P e96c75365fd245a99b23e1d9c2f7d58b [term 1 FOLLOWER]: Refusing update from remote peer d9f5760258c741e7825ba15a5650e3d2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 16:24:22.433763 18671 raft_consensus.cc:1261] T 5f2e44295ade4a2490f1234f5469bf04 P 7d2b62849605417c8faf84efe9ffdfcb [term 1 FOLLOWER]: Refusing update from remote peer d9f5760258c741e7825ba15a5650e3d2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 16:24:22.434105 18775 consensus_queue.cc:1035] T 5f2e44295ade4a2490f1234f5469bf04 P d9f5760258c741e7825ba15a5650e3d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d2b62849605417c8faf84efe9ffdfcb" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 39375 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 16:24:22.434281 18781 consensus_queue.cc:1035] T 5f2e44295ade4a2490f1234f5469bf04 P d9f5760258c741e7825ba15a5650e3d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e96c75365fd245a99b23e1d9c2f7d58b" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 45995 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 16:24:22.437379 18786 mvcc.cc:205] Tried to move back new op lower bound from 7316654745325678592 to 7316654745175695360. Current Snapshot: MvccSnapshot[applied={T|T < 7316654745325678592}]
I20260809 16:24:22.438714 18081 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:55942:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 16:24:22.438824 18081 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:55942:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 16:24:22.439572 18081 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6b8a856b37a24e5f80d1a8c25dd724c9: Sending DeleteTablet for 3 replicas of tablet 5f2e44295ade4a2490f1234f5469bf04
I20260809 16:24:22.439888 18454 tablet_service.cc:1514] Processing DeleteTablet for tablet 5f2e44295ade4a2490f1234f5469bf04 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:24:22 UTC) from {username='slave'} at 127.0.0.1:45260
I20260809 16:24:22.439919 18270 tablet_service.cc:1514] Processing DeleteTablet for tablet 5f2e44295ade4a2490f1234f5469bf04 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:24:22 UTC) from {username='slave'} at 127.0.0.1:59844
I20260809 16:24:22.439936 18641 tablet_service.cc:1514] Processing DeleteTablet for tablet 5f2e44295ade4a2490f1234f5469bf04 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:24:22 UTC) from {username='slave'} at 127.0.0.1:53762
I20260809 16:24:22.440119 18797 tablet_replica.cc:330] T 5f2e44295ade4a2490f1234f5469bf04 P d9f5760258c741e7825ba15a5650e3d2: stopping tablet replica
I20260809 16:24:22.440212 18797 raft_consensus.cc:2229] T 5f2e44295ade4a2490f1234f5469bf04 P d9f5760258c741e7825ba15a5650e3d2 [term 1 LEADER]: Raft consensus shutting down.
I20260809 16:24:22.440315 18797 raft_consensus.cc:2258] T 5f2e44295ade4a2490f1234f5469bf04 P d9f5760258c741e7825ba15a5650e3d2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 16:24:22.440449 21710 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrNqUCn/build/release/bin/kudu with pid 18169
I20260809 16:24:22.449468 18797 ts_tablet_manager.cc:1905] T 5f2e44295ade4a2490f1234f5469bf04 P d9f5760258c741e7825ba15a5650e3d2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 16:24:22.451967 18797 ts_tablet_manager.cc:1918] T 5f2e44295ade4a2490f1234f5469bf04 P d9f5760258c741e7825ba15a5650e3d2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 16:24:22.452059 18797 log.cc:1196] T 5f2e44295ade4a2490f1234f5469bf04 P d9f5760258c741e7825ba15a5650e3d2: Deleting WAL directory at /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786292417722232-21710-0/minicluster-data/ts-1/wal/wals/5f2e44295ade4a2490f1234f5469bf04
I20260809 16:24:22.452391 18797 ts_tablet_manager.cc:1939] T 5f2e44295ade4a2490f1234f5469bf04 P d9f5760258c741e7825ba15a5650e3d2: Deleting consensus metadata
I20260809 16:24:22.452922 18051 catalog_manager.cc:4841] TS d9f5760258c741e7825ba15a5650e3d2 (127.21.51.130:45635): tablet 5f2e44295ade4a2490f1234f5469bf04 (table test-table [id=afcea12d64974f8088bc907df4c2adac]) successfully deleted
I20260809 16:24:22.469384 18799 tablet_replica.cc:330] T 5f2e44295ade4a2490f1234f5469bf04 P 7d2b62849605417c8faf84efe9ffdfcb: stopping tablet replica
W20260809 16:24:22.473731 18051 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.21.51.129:45995 (error 108)
I20260809 16:24:22.473868 18799 raft_consensus.cc:2229] T 5f2e44295ade4a2490f1234f5469bf04 P 7d2b62849605417c8faf84efe9ffdfcb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 16:24:22.474093 18799 raft_consensus.cc:2258] T 5f2e44295ade4a2490f1234f5469bf04 P 7d2b62849605417c8faf84efe9ffdfcb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 16:24:22.474346 18756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:22.474198 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.129:45995 (local address 127.0.0.1:59860)
0809 16:24:22.474260 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0809 16:24:22.474307 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.51.129:45995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260809 16:24:22.474507 18051 catalog_manager.cc:4584] TS e96c75365fd245a99b23e1d9c2f7d58b (127.21.51.129:45995): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 5f2e44295ade4a2490f1234f5469bf04: Network error: Client connection negotiation failed: client connection to 127.21.51.129:45995: connect: Connection refused (error 111)
I20260809 16:24:22.474663 18799 ts_tablet_manager.cc:1905] T 5f2e44295ade4a2490f1234f5469bf04 P 7d2b62849605417c8faf84efe9ffdfcb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 16:24:22.474947 21710 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrNqUCn/build/release/bin/kudu with pid 18359
I20260809 16:24:22.480000 18799 ts_tablet_manager.cc:1918] T 5f2e44295ade4a2490f1234f5469bf04 P 7d2b62849605417c8faf84efe9ffdfcb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 16:24:22.480108 18799 log.cc:1196] T 5f2e44295ade4a2490f1234f5469bf04 P 7d2b62849605417c8faf84efe9ffdfcb: Deleting WAL directory at /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786292417722232-21710-0/minicluster-data/ts-2/wal/wals/5f2e44295ade4a2490f1234f5469bf04
I20260809 16:24:22.480386 18799 ts_tablet_manager.cc:1939] T 5f2e44295ade4a2490f1234f5469bf04 P 7d2b62849605417c8faf84efe9ffdfcb: Deleting consensus metadata
I20260809 16:24:22.480975 18058 catalog_manager.cc:4841] TS 7d2b62849605417c8faf84efe9ffdfcb (127.21.51.131:39375): tablet 5f2e44295ade4a2490f1234f5469bf04 (table test-table [id=afcea12d64974f8088bc907df4c2adac]) successfully deleted
I20260809 16:24:22.484354 21710 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrNqUCn/build/release/bin/kudu with pid 18556
I20260809 16:24:22.491024 21710 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrNqUCn/build/release/bin/kudu with pid 18029
2026-08-09T16:24:22Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (1097 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786292417722232-21710-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 16:24:22 dist-test-slave-s11t krb5kdc[18803](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 16:24:22 dist-test-slave-s11t krb5kdc[18803](info): set up 2 sockets
Aug 09 16:24:22 dist-test-slave-s11t krb5kdc[18803](info): commencing operation
krb5kdc: starting...
W20260809 16:24:23.733743 21710 mini_kdc.cc:121] Time spent starting KDC: real 1.218s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786292417722232-21710-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786292417722232-21710-0/krb5kdc/test-user.keytab.
Aug 09 16:24:23 dist-test-slave-s11t krb5kdc[18803](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292663, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T16:24:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T16:24:23Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.51.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.51.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.51.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.190.keytab.
Entry for principal kudu/127.21.51.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.190.keytab.
I20260809 16:24:23.853492 21710 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrNqUCn/build/release/bin/kudu
/tmp/dist-test-taskrNqUCn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786292417722232-21710-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786292417722232-21710-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786292417722232-21710-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786292417722232-21710-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.51.190:38301
--webserver_interface=127.21.51.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.190.keytab
--principal=kudu/127.21.51.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.51.148:36051
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.51.190:38301
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786292417722232-21710-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786292417722232-21710-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786292417722232-21710-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:24:23.934688 18826 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:24:23.934895 18826 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:24:23.934921 18826 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:24:23.936007 18826 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 16:24:23.936043 18826 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:24:23.936058 18826 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:24:23.936070 18826 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 16:24:23.936082 18826 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 16:24:23.937513 18826 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.51.148:36051
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786292417722232-21710-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786292417722232-21710-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.51.190:38301
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.190.keytab
--principal=kudu/127.21.51.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.51.190:38301
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786292417722232-21710-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.51.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18826
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786292417722232-21710-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:24:23.937866 18826 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:24:23.938117 18826 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:24:23.943192 18831 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:24:23.943171 18834 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:24:23.943167 18832 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:24:23.943344 18826 server_base.cc:1034] running on GCE node
I20260809 16:24:23.943665 18826 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:24:23.943954 18826 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:24:23.945134 18826 hybrid_clock.cc:648] HybridClock initialized: now 1786292663945125 us; error 44 us; skew 500 ppm
Aug 09 16:24:23 dist-test-slave-s11t krb5kdc[18803](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292663, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.51.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:24:23.947602 18826 init.cc:377] Logged in from keytab as kudu/127.21.51.190@KRBTEST.COM (short username kudu)
I20260809 16:24:23.948094 18826 webserver.cc:460] Webserver started at http://127.21.51.190:45693/ using document root <none> and password file <none>
I20260809 16:24:23.948321 18826 fs_manager.cc:359] Metadata directory not provided
I20260809 16:24:23.948392 18826 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:24:23.948552 18826 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:24:23.949693 18826 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786292417722232-21710-0/minicluster-data/master-0/data/instance:
uuid: "355e97e0fa674a6e84e7155fb4618297"
format_stamp: "Formatted at 2026-08-09 16:24:23 on dist-test-slave-s11t"
server_key: "0a6698011d3a0563097db57819d1caf3"
server_key_iv: "8c1feaec3346712415b5c3dd9d2196c9"
server_key_version: "encryptionkey@0"
I20260809 16:24:23.950070 18826 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786292417722232-21710-0/minicluster-data/master-0/wal/instance:
uuid: "355e97e0fa674a6e84e7155fb4618297"
format_stamp: "Formatted at 2026-08-09 16:24:23 on dist-test-slave-s11t"
server_key: "0a6698011d3a0563097db57819d1caf3"
server_key_iv: "8c1feaec3346712415b5c3dd9d2196c9"
server_key_version: "encryptionkey@0"
I20260809 16:24:23.951467 18826 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 16:24:23.952311 18848 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:24:23.952538 18826 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:24:23.952643 18826 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786292417722232-21710-0/minicluster-data/master-0/data,/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786292417722232-21710-0/minicluster-data/master-0/wal
uuid: "355e97e0fa674a6e84e7155fb4618297"
format_stamp: "Formatted at 2026-08-09 16:24:23 on dist-test-slave-s11t"
server_key: "0a6698011d3a0563097db57819d1caf3"
server_key_iv: "8c1feaec3346712415b5c3dd9d2196c9"
server_key_version: "encryptionkey@0"
I20260809 16:24:23.952759 18826 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786292417722232-21710-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786292417722232-21710-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786292417722232-21710-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:24:23.969182 18826 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:24:23.971163 18826 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:24:23.971343 18826 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:24:23.975703 18826 rpc_server.cc:307] RPC server started. Bound to: 127.21.51.190:38301
I20260809 16:24:23.975740 18921 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.51.190:38301 every 8 connection(s)
I20260809 16:24:23.976086 18826 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786292417722232-21710-0/minicluster-data/master-0/data/info.pb
I20260809 16:24:23.977033 18923 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:24:23.978317 21710 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrNqUCn/build/release/bin/kudu as pid 18826
I20260809 16:24:23.978446 21710 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786292417722232-21710-0/minicluster-data/master-0/wal/instance
I20260809 16:24:23.978677 21710 external_mini_cluster.cc:1428] Setting key 204cb22b37102f4923579f5233fbe0d9
I20260809 16:24:23.979882 18923 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 355e97e0fa674a6e84e7155fb4618297: Bootstrap starting.
I20260809 16:24:23.980480 18923 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 355e97e0fa674a6e84e7155fb4618297: Neither blocks nor log segments found. Creating new log.
I20260809 16:24:23.980836 18923 log.cc:824] T 00000000000000000000000000000000 P 355e97e0fa674a6e84e7155fb4618297: Log is configured to *not* fsync() on all Append() calls
I20260809 16:24:23.981549 18923 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 355e97e0fa674a6e84e7155fb4618297: No bootstrap required, opened a new log
I20260809 16:24:23.983309 18923 raft_consensus.cc:348] T 00000000000000000000000000000000 P 355e97e0fa674a6e84e7155fb4618297 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "355e97e0fa674a6e84e7155fb4618297" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 38301 } }
I20260809 16:24:23.983479 18923 raft_consensus.cc:374] T 00000000000000000000000000000000 P 355e97e0fa674a6e84e7155fb4618297 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:24:23.983526 18923 raft_consensus.cc:729] T 00000000000000000000000000000000 P 355e97e0fa674a6e84e7155fb4618297 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 355e97e0fa674a6e84e7155fb4618297, State: Initialized, Role: FOLLOWER
I20260809 16:24:23.983690 18923 consensus_queue.cc:260] T 00000000000000000000000000000000 P 355e97e0fa674a6e84e7155fb4618297 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "355e97e0fa674a6e84e7155fb4618297" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 38301 } }
I20260809 16:24:23.983784 18923 raft_consensus.cc:388] T 00000000000000000000000000000000 P 355e97e0fa674a6e84e7155fb4618297 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 16:24:23.983820 18923 raft_consensus.cc:482] T 00000000000000000000000000000000 P 355e97e0fa674a6e84e7155fb4618297 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 16:24:23.983876 18923 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 355e97e0fa674a6e84e7155fb4618297 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:24:23.984596 18923 raft_consensus.cc:504] T 00000000000000000000000000000000 P 355e97e0fa674a6e84e7155fb4618297 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "355e97e0fa674a6e84e7155fb4618297" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 38301 } }
I20260809 16:24:23.984751 18923 leader_election.cc:302] T 00000000000000000000000000000000 P 355e97e0fa674a6e84e7155fb4618297 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 355e97e0fa674a6e84e7155fb4618297; no voters:
I20260809 16:24:23.984932 18923 leader_election.cc:288] T 00000000000000000000000000000000 P 355e97e0fa674a6e84e7155fb4618297 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 16:24:23.984969 18929 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 355e97e0fa674a6e84e7155fb4618297 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:24:23.985185 18929 raft_consensus.cc:686] T 00000000000000000000000000000000 P 355e97e0fa674a6e84e7155fb4618297 [term 1 LEADER]: Becoming Leader. State: Replica: 355e97e0fa674a6e84e7155fb4618297, State: Running, Role: LEADER
I20260809 16:24:23.985200 18923 sys_catalog.cc:564] T 00000000000000000000000000000000 P 355e97e0fa674a6e84e7155fb4618297 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 16:24:23.985404 18929 consensus_queue.cc:237] T 00000000000000000000000000000000 P 355e97e0fa674a6e84e7155fb4618297 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "355e97e0fa674a6e84e7155fb4618297" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 38301 } }
I20260809 16:24:23.985759 18932 sys_catalog.cc:455] T 00000000000000000000000000000000 P 355e97e0fa674a6e84e7155fb4618297 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "355e97e0fa674a6e84e7155fb4618297" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "355e97e0fa674a6e84e7155fb4618297" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 38301 } } }
I20260809 16:24:23.985872 18932 sys_catalog.cc:458] T 00000000000000000000000000000000 P 355e97e0fa674a6e84e7155fb4618297 [sys.catalog]: This master's current role is: LEADER
I20260809 16:24:23.986099 18941 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
Aug 09 16:24:23 dist-test-slave-s11t krb5kdc[18803](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292663, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.51.190@KRBTEST.COM
I20260809 16:24:23.986617 18941 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 16:24:23.987082 18930 sys_catalog.cc:455] T 00000000000000000000000000000000 P 355e97e0fa674a6e84e7155fb4618297 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 355e97e0fa674a6e84e7155fb4618297. Latest consensus state: current_term: 1 leader_uuid: "355e97e0fa674a6e84e7155fb4618297" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "355e97e0fa674a6e84e7155fb4618297" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 38301 } } }
I20260809 16:24:23.987317 18930 sys_catalog.cc:458] T 00000000000000000000000000000000 P 355e97e0fa674a6e84e7155fb4618297 [sys.catalog]: This master's current role is: LEADER
I20260809 16:24:23.988451 18941 catalog_manager.cc:1347] Generated new cluster ID: dbd46939f0694f61820fcd505ef83af0
I20260809 16:24:23.988510 18941 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 16:24:23.990571 18926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:23.979225 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42174 (local address 127.21.51.190:38301)
0809 16:24:23.979419 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0809 16:24:23.979427 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 16:24:23.979452 (+ 25us) server_negotiation.cc:407] Connection header received
0809 16:24:23.979846 (+ 394us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:24:23.979851 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:24:23.979894 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:24:23.980231 (+ 337us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:24:23.981107 (+ 876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:24:23.982168 (+ 1061us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:24:23.983035 (+ 867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:24:23.983557 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:24:23.983681 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:24:23.986947 (+ 3266us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:24:23.986952 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:24:23.986953 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:24:23.986967 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:24:23.989029 (+ 2062us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:24:23.989441 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:24:23.989443 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:24:23.989443 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:24:23.989536 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:24:23.989750 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:24:23.989750 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:24:23.989751 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:24:23.989994 (+ 243us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:24:23.990093 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0809 16:24:23.990239 (+ 146us) server_negotiation.cc:299] Negotiation successful
0809 16:24:23.990338 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":136,"thread_start_us":96,"threads_started":1}
I20260809 16:24:24.012126 18941 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 16:24:24.012933 18941 catalog_manager.cc:1504] Loading token signing keys...
I20260809 16:24:24.019025 18941 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 355e97e0fa674a6e84e7155fb4618297: Generated new TSK 0
I20260809 16:24:24.019312 18941 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.21.51.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.51.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.51.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.129.keytab.
Entry for principal kudu/127.21.51.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.129.keytab.
I20260809 16:24:24.057549 21710 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrNqUCn/build/release/bin/kudu
/tmp/dist-test-taskrNqUCn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786292417722232-21710-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786292417722232-21710-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786292417722232-21710-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786292417722232-21710-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.51.129:0
--local_ip_for_outbound_sockets=127.21.51.129
--webserver_interface=127.21.51.129
--webserver_port=0
--tserver_master_addrs=127.21.51.190:38301
--keytab_file=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.129.keytab
--principal=kudu/127.21.51.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.51.148:36051
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786292417722232-21710-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786292417722232-21710-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786292417722232-21710-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:24:24.139720 18958 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:24:24.139917 18958 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:24:24.139940 18958 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:24:24.141080 18958 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:24:24.141122 18958 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:24:24.141148 18958 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.51.129
I20260809 16:24:24.142729 18958 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.51.148:36051
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786292417722232-21710-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786292417722232-21710-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.129.keytab
--principal=kudu/127.21.51.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.51.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786292417722232-21710-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.51.129
--webserver_port=0
--tserver_master_addrs=127.21.51.190:38301
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18958
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.51.129
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786292417722232-21710-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:24:24.142988 18958 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:24:24.143217 18958 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:24:24.148285 18976 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:24:24.148283 18971 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:24:24.148350 18966 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:24:24.148778 18958 server_base.cc:1034] running on GCE node
I20260809 16:24:24.148964 18958 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:24:24.149165 18958 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:24:24.150308 18958 hybrid_clock.cc:648] HybridClock initialized: now 1786292664150298 us; error 28 us; skew 500 ppm
Aug 09 16:24:24 dist-test-slave-s11t krb5kdc[18803](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292664, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.51.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:24:24.152952 18958 init.cc:377] Logged in from keytab as kudu/127.21.51.129@KRBTEST.COM (short username kudu)
I20260809 16:24:24.153384 18958 webserver.cc:460] Webserver started at http://127.21.51.129:46509/ using document root <none> and password file <none>
I20260809 16:24:24.153605 18958 fs_manager.cc:359] Metadata directory not provided
I20260809 16:24:24.153650 18958 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:24:24.153795 18958 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:24:24.154829 18958 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786292417722232-21710-0/minicluster-data/ts-0/data/instance:
uuid: "59815c45af0f45f69215e4b9f19eec0f"
format_stamp: "Formatted at 2026-08-09 16:24:24 on dist-test-slave-s11t"
server_key: "29d6a87088ce2b50aee9d637ffad5315"
server_key_iv: "4f221a008ae381ae36e88b34f378b20d"
server_key_version: "encryptionkey@0"
I20260809 16:24:24.155200 18958 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786292417722232-21710-0/minicluster-data/ts-0/wal/instance:
uuid: "59815c45af0f45f69215e4b9f19eec0f"
format_stamp: "Formatted at 2026-08-09 16:24:24 on dist-test-slave-s11t"
server_key: "29d6a87088ce2b50aee9d637ffad5315"
server_key_iv: "4f221a008ae381ae36e88b34f378b20d"
server_key_version: "encryptionkey@0"
I20260809 16:24:24.156555 18958 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260809 16:24:24.157388 18990 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:24:24.157572 18958 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 16:24:24.157665 18958 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786292417722232-21710-0/minicluster-data/ts-0/data,/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786292417722232-21710-0/minicluster-data/ts-0/wal
uuid: "59815c45af0f45f69215e4b9f19eec0f"
format_stamp: "Formatted at 2026-08-09 16:24:24 on dist-test-slave-s11t"
server_key: "29d6a87088ce2b50aee9d637ffad5315"
server_key_iv: "4f221a008ae381ae36e88b34f378b20d"
server_key_version: "encryptionkey@0"
I20260809 16:24:24.157739 18958 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786292417722232-21710-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786292417722232-21710-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786292417722232-21710-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:24:24.182035 18958 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:24:24.184456 18958 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:24:24.184629 18958 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:24:24.184923 18958 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:24:24.185310 18958 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:24:24.185382 18958 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:24:24.185422 18958 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:24:24.185446 18958 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:24:24.193579 18958 rpc_server.cc:307] RPC server started. Bound to: 127.21.51.129:32913
I20260809 16:24:24.193641 19157 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.51.129:32913 every 8 connection(s)
I20260809 16:24:24.194003 18958 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786292417722232-21710-0/minicluster-data/ts-0/data/info.pb
Aug 09 16:24:24 dist-test-slave-s11t krb5kdc[18803](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292664, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.51.129@KRBTEST.COM for kudu/127.21.51.190@KRBTEST.COM
I20260809 16:24:24.202947 18926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:24.194807 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.51.129:52563 (local address 127.21.51.190:38301)
0809 16:24:24.194882 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0809 16:24:24.194884 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:24:24.195101 (+ 217us) server_negotiation.cc:407] Connection header received
0809 16:24:24.195549 (+ 448us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:24:24.195550 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:24:24.195561 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:24:24.195614 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:24:24.196516 (+ 902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:24:24.197047 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:24:24.197996 (+ 949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:24:24.198196 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:24:24.198247 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:24:24.200262 (+ 2015us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:24:24.200264 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:24:24.200264 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:24:24.200271 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:24:24.201724 (+ 1453us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:24:24.202088 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:24:24.202089 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:24:24.202089 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:24:24.202140 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:24:24.202356 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:24:24.202356 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:24:24.202357 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:24:24.202456 (+ 99us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:24:24.202507 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0809 16:24:24.202782 (+ 275us) server_negotiation.cc:299] Negotiation successful
0809 16:24:24.202856 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260809 16:24:24.203099 19162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:24.194785 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:38301 (local address 127.21.51.129:52563)
0809 16:24:24.194973 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0809 16:24:24.195009 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0809 16:24:24.195468 (+ 459us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:24:24.195665 (+ 197us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:24:24.195669 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:24:24.196242 (+ 573us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:24:24.196436 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:24:24.196441 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:24:24.197124 (+ 683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:24:24.197125 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:24:24.197920 (+ 795us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:24:24.197922 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:24:24.198269 (+ 347us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:24:24.198270 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:24:24.198378 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:24:24.198814 (+ 436us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:24:24.198822 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:24:24.200172 (+ 1350us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:24:24.201792 (+ 1620us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:24:24.201798 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:24:24.201799 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:24:24.202021 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:24:24.202198 (+ 177us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:24:24.202198 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:24:24.202198 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:24:24.202302 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:24:24.202539 (+ 237us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:24:24.202543 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:24:24.202656 (+ 113us) client_negotiation.cc:769] Sending connection context
0809 16:24:24.202751 (+ 95us) client_negotiation.cc:240] Negotiation successful
0809 16:24:24.202870 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":89,"threads_started":1}
I20260809 16:24:24.203390 21710 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrNqUCn/build/release/bin/kudu as pid 18958
I20260809 16:24:24.203542 21710 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786292417722232-21710-0/minicluster-data/ts-0/wal/instance
I20260809 16:24:24.203678 19159 heartbeater.cc:347] Connected to a master server at 127.21.51.190:38301
I20260809 16:24:24.203747 21710 external_mini_cluster.cc:1428] Setting key 03fc825aa2e4017a84c3fc1dd587793f
I20260809 16:24:24.203799 19159 heartbeater.cc:464] Registering TS with master...
I20260809 16:24:24.204000 19159 heartbeater.cc:511] Master 127.21.51.190:38301 requested a full tablet report, sending...
I20260809 16:24:24.204584 18878 ts_manager.cc:194] Registered new tserver with Master: 59815c45af0f45f69215e4b9f19eec0f (127.21.51.129:32913)
I20260809 16:24:24.205487 18878 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.51.129@KRBTEST.COM'} at 127.21.51.129:52563
WARNING: no policy specified for kudu/127.21.51.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.51.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.51.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.130.keytab.
Entry for principal kudu/127.21.51.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.130.keytab.
I20260809 16:24:24.229243 21710 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrNqUCn/build/release/bin/kudu
/tmp/dist-test-taskrNqUCn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786292417722232-21710-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786292417722232-21710-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786292417722232-21710-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786292417722232-21710-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.51.130:0
--local_ip_for_outbound_sockets=127.21.51.130
--webserver_interface=127.21.51.130
--webserver_port=0
--tserver_master_addrs=127.21.51.190:38301
--keytab_file=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.130.keytab
--principal=kudu/127.21.51.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.51.148:36051
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786292417722232-21710-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786292417722232-21710-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786292417722232-21710-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:24:24.314997 19166 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:24:24.315215 19166 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:24:24.315253 19166 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:24:24.316421 19166 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:24:24.316488 19166 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:24:24.316534 19166 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.51.130
I20260809 16:24:24.318050 19166 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.51.148:36051
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786292417722232-21710-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786292417722232-21710-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.130.keytab
--principal=kudu/127.21.51.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.51.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786292417722232-21710-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.51.130
--webserver_port=0
--tserver_master_addrs=127.21.51.190:38301
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19166
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.51.130
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786292417722232-21710-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:24:24.318320 19166 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:24:24.318564 19166 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:24:24.323685 19172 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:24:24.323697 19178 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:24:24.323685 19175 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:24:24.323901 19166 server_base.cc:1034] running on GCE node
I20260809 16:24:24.324159 19166 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:24:24.324402 19166 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:24:24.325567 19166 hybrid_clock.cc:648] HybridClock initialized: now 1786292664325557 us; error 31 us; skew 500 ppm
Aug 09 16:24:24 dist-test-slave-s11t krb5kdc[18803](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292664, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.51.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:24:24.328030 19166 init.cc:377] Logged in from keytab as kudu/127.21.51.130@KRBTEST.COM (short username kudu)
I20260809 16:24:24.328438 19166 webserver.cc:460] Webserver started at http://127.21.51.130:37137/ using document root <none> and password file <none>
I20260809 16:24:24.328720 19166 fs_manager.cc:359] Metadata directory not provided
I20260809 16:24:24.328799 19166 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:24:24.328965 19166 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:24:24.329955 19166 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786292417722232-21710-0/minicluster-data/ts-1/data/instance:
uuid: "b863f9eba8a94b1282a2e78435824cf9"
format_stamp: "Formatted at 2026-08-09 16:24:24 on dist-test-slave-s11t"
server_key: "1b0b759859cc5d74f9f20614bd2200eb"
server_key_iv: "2c83de5d738631e92ca0e4fd215eb28b"
server_key_version: "encryptionkey@0"
I20260809 16:24:24.330319 19166 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786292417722232-21710-0/minicluster-data/ts-1/wal/instance:
uuid: "b863f9eba8a94b1282a2e78435824cf9"
format_stamp: "Formatted at 2026-08-09 16:24:24 on dist-test-slave-s11t"
server_key: "1b0b759859cc5d74f9f20614bd2200eb"
server_key_iv: "2c83de5d738631e92ca0e4fd215eb28b"
server_key_version: "encryptionkey@0"
I20260809 16:24:24.331640 19166 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 16:24:24.332330 19186 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:24:24.332515 19166 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 16:24:24.332614 19166 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786292417722232-21710-0/minicluster-data/ts-1/data,/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786292417722232-21710-0/minicluster-data/ts-1/wal
uuid: "b863f9eba8a94b1282a2e78435824cf9"
format_stamp: "Formatted at 2026-08-09 16:24:24 on dist-test-slave-s11t"
server_key: "1b0b759859cc5d74f9f20614bd2200eb"
server_key_iv: "2c83de5d738631e92ca0e4fd215eb28b"
server_key_version: "encryptionkey@0"
I20260809 16:24:24.332721 19166 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786292417722232-21710-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786292417722232-21710-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786292417722232-21710-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:24:24.362324 19166 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:24:24.364384 19166 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:24:24.364557 19166 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:24:24.364853 19166 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:24:24.365192 19166 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:24:24.365247 19166 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:24:24.365283 19166 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:24:24.365310 19166 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:24:24.370858 19166 rpc_server.cc:307] RPC server started. Bound to: 127.21.51.130:34939
I20260809 16:24:24.370946 19369 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.51.130:34939 every 8 connection(s)
I20260809 16:24:24.371265 19166 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786292417722232-21710-0/minicluster-data/ts-1/data/info.pb
I20260809 16:24:24.374172 21710 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrNqUCn/build/release/bin/kudu as pid 19166
I20260809 16:24:24.374308 21710 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786292417722232-21710-0/minicluster-data/ts-1/wal/instance
I20260809 16:24:24.374564 21710 external_mini_cluster.cc:1428] Setting key 31215fb273e6775ed3d82c3e97082ac1
Aug 09 16:24:24 dist-test-slave-s11t krb5kdc[18803](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292664, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.51.130@KRBTEST.COM for kudu/127.21.51.190@KRBTEST.COM
I20260809 16:24:24.381151 18926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:24.372019 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.51.130:44021 (local address 127.21.51.190:38301)
0809 16:24:24.372065 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0809 16:24:24.372067 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:24:24.372259 (+ 192us) server_negotiation.cc:407] Connection header received
0809 16:24:24.372743 (+ 484us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:24:24.372744 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:24:24.372755 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:24:24.372823 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:24:24.373755 (+ 932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:24:24.374340 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:24:24.375638 (+ 1298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:24:24.375941 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:24:24.376031 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:24:24.378186 (+ 2155us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:24:24.378188 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:24:24.378189 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:24:24.378197 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:24:24.379659 (+ 1462us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:24:24.380095 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:24:24.380097 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:24:24.380097 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:24:24.380163 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:24:24.380461 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:24:24.380462 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:24:24.380462 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:24:24.380578 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:24:24.380640 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0809 16:24:24.381014 (+ 374us) server_negotiation.cc:299] Negotiation successful
0809 16:24:24.381087 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260809 16:24:24.381459 19373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:24.371960 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:38301 (local address 127.21.51.130:44021)
0809 16:24:24.372120 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0809 16:24:24.372161 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0809 16:24:24.372627 (+ 466us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:24:24.372875 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:24:24.372879 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:24:24.373465 (+ 586us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:24:24.373657 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:24:24.373662 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:24:24.374459 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:24:24.374460 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:24:24.375534 (+ 1074us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:24:24.375536 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:24:24.376038 (+ 502us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:24:24.376039 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:24:24.376137 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:24:24.376651 (+ 514us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:24:24.376658 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:24:24.378086 (+ 1428us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:24:24.379770 (+ 1684us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:24:24.379777 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:24:24.379780 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:24:24.380013 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:24:24.380235 (+ 222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:24:24.380236 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:24:24.380236 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:24:24.380387 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:24:24.380656 (+ 269us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:24:24.380663 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:24:24.380856 (+ 193us) client_negotiation.cc:769] Sending connection context
0809 16:24:24.380980 (+ 124us) client_negotiation.cc:240] Negotiation successful
0809 16:24:24.381169 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":91,"threads_started":1}
I20260809 16:24:24.382015 19370 heartbeater.cc:347] Connected to a master server at 127.21.51.190:38301
I20260809 16:24:24.382108 19370 heartbeater.cc:464] Registering TS with master...
I20260809 16:24:24.382304 19370 heartbeater.cc:511] Master 127.21.51.190:38301 requested a full tablet report, sending...
I20260809 16:24:24.382738 18878 ts_manager.cc:194] Registered new tserver with Master: b863f9eba8a94b1282a2e78435824cf9 (127.21.51.130:34939)
I20260809 16:24:24.383095 18878 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.51.130@KRBTEST.COM'} at 127.21.51.130:44021
WARNING: no policy specified for kudu/127.21.51.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.51.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.51.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.131.keytab.
Entry for principal kudu/127.21.51.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.131.keytab.
I20260809 16:24:24.401117 21710 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrNqUCn/build/release/bin/kudu
/tmp/dist-test-taskrNqUCn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786292417722232-21710-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786292417722232-21710-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786292417722232-21710-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786292417722232-21710-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.51.131:0
--local_ip_for_outbound_sockets=127.21.51.131
--webserver_interface=127.21.51.131
--webserver_port=0
--tserver_master_addrs=127.21.51.190:38301
--keytab_file=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.131.keytab
--principal=kudu/127.21.51.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.51.148:36051
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786292417722232-21710-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786292417722232-21710-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786292417722232-21710-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:24:24.484139 19378 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:24:24.484340 19378 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:24:24.484364 19378 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:24:24.485620 19378 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:24:24.485661 19378 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:24:24.485689 19378 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.51.131
I20260809 16:24:24.487308 19378 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.51.148:36051
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786292417722232-21710-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786292417722232-21710-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.131.keytab
--principal=kudu/127.21.51.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.51.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786292417722232-21710-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.51.131
--webserver_port=0
--tserver_master_addrs=127.21.51.190:38301
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19378
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.51.131
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786292417722232-21710-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:24:24.487553 19378 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:24:24.487785 19378 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:24:24.492871 19386 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:24:24.492981 19378 server_base.cc:1034] running on GCE node
W20260809 16:24:24.493021 19388 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:24:24.492901 19394 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:24:24.493219 19378 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:24:24.493465 19378 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:24:24.494626 19378 hybrid_clock.cc:648] HybridClock initialized: now 1786292664494604 us; error 53 us; skew 500 ppm
Aug 09 16:24:24 dist-test-slave-s11t krb5kdc[18803](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292664, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.51.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:24:24.497160 19378 init.cc:377] Logged in from keytab as kudu/127.21.51.131@KRBTEST.COM (short username kudu)
I20260809 16:24:24.497671 19378 webserver.cc:460] Webserver started at http://127.21.51.131:33937/ using document root <none> and password file <none>
I20260809 16:24:24.497920 19378 fs_manager.cc:359] Metadata directory not provided
I20260809 16:24:24.497967 19378 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:24:24.498126 19378 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:24:24.499125 19378 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786292417722232-21710-0/minicluster-data/ts-2/data/instance:
uuid: "d0c6d9dbe88d4c97b50835a135d64955"
format_stamp: "Formatted at 2026-08-09 16:24:24 on dist-test-slave-s11t"
server_key: "372b2df73e634979d3f56c862cd5084f"
server_key_iv: "403f7105e87560c6ba99ae6abe1674d8"
server_key_version: "encryptionkey@0"
I20260809 16:24:24.499486 19378 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786292417722232-21710-0/minicluster-data/ts-2/wal/instance:
uuid: "d0c6d9dbe88d4c97b50835a135d64955"
format_stamp: "Formatted at 2026-08-09 16:24:24 on dist-test-slave-s11t"
server_key: "372b2df73e634979d3f56c862cd5084f"
server_key_iv: "403f7105e87560c6ba99ae6abe1674d8"
server_key_version: "encryptionkey@0"
I20260809 16:24:24.500837 19378 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 16:24:24.501569 19402 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:24:24.501776 19378 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:24:24.501880 19378 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786292417722232-21710-0/minicluster-data/ts-2/data,/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786292417722232-21710-0/minicluster-data/ts-2/wal
uuid: "d0c6d9dbe88d4c97b50835a135d64955"
format_stamp: "Formatted at 2026-08-09 16:24:24 on dist-test-slave-s11t"
server_key: "372b2df73e634979d3f56c862cd5084f"
server_key_iv: "403f7105e87560c6ba99ae6abe1674d8"
server_key_version: "encryptionkey@0"
I20260809 16:24:24.501959 19378 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786292417722232-21710-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786292417722232-21710-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786292417722232-21710-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:24:24.508141 19378 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:24:24.509930 19378 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:24:24.510087 19378 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:24:24.510301 19378 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:24:24.510586 19378 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:24:24.510639 19378 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:24:24.510679 19378 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:24:24.510705 19378 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:24:24.516491 19378 rpc_server.cc:307] RPC server started. Bound to: 127.21.51.131:46065
I20260809 16:24:24.516534 19573 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.51.131:46065 every 8 connection(s)
I20260809 16:24:24.516943 19378 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786292417722232-21710-0/minicluster-data/ts-2/data/info.pb
Aug 09 16:24:24 dist-test-slave-s11t krb5kdc[18803](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292664, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.51.131@KRBTEST.COM for kudu/127.21.51.190@KRBTEST.COM
I20260809 16:24:24.525954 21710 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrNqUCn/build/release/bin/kudu as pid 19378
I20260809 16:24:24.526105 21710 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786292417722232-21710-0/minicluster-data/ts-2/wal/instance
I20260809 16:24:24.526345 21710 external_mini_cluster.cc:1428] Setting key 1d0107dd14496353f9df46ac06ff2265
I20260809 16:24:24.526588 18926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:24.517651 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.51.131:56923 (local address 127.21.51.190:38301)
0809 16:24:24.517732 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0809 16:24:24.517735 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 16:24:24.517898 (+ 163us) server_negotiation.cc:407] Connection header received
0809 16:24:24.518336 (+ 438us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:24:24.518338 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:24:24.518348 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:24:24.518398 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:24:24.519378 (+ 980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:24:24.520029 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:24:24.521104 (+ 1075us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:24:24.521306 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:24:24.521374 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:24:24.523664 (+ 2290us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:24:24.523665 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:24:24.523665 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:24:24.523672 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:24:24.525101 (+ 1429us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:24:24.525497 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:24:24.525498 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:24:24.525498 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:24:24.525549 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:24:24.525820 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:24:24.525821 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:24:24.525821 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:24:24.525936 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:24:24.525987 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0809 16:24:24.526377 (+ 390us) server_negotiation.cc:299] Negotiation successful
0809 16:24:24.526478 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260809 16:24:24.526834 19576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:24.517606 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:38301 (local address 127.21.51.131:56923)
0809 16:24:24.517783 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0809 16:24:24.517817 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0809 16:24:24.518253 (+ 436us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:24:24.518457 (+ 204us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:24:24.518459 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:24:24.519110 (+ 651us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:24:24.519295 (+ 185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:24:24.519300 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:24:24.520127 (+ 827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:24:24.520129 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:24:24.521006 (+ 877us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:24:24.521009 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:24:24.521402 (+ 393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:24:24.521403 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:24:24.521505 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:24:24.522022 (+ 517us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:24:24.522031 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:24:24.523560 (+ 1529us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:24:24.525177 (+ 1617us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:24:24.525183 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:24:24.525185 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:24:24.525421 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:24:24.525611 (+ 190us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:24:24.525611 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:24:24.525612 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:24:24.525714 (+ 102us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:24:24.526008 (+ 294us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:24:24.526015 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:24:24.526216 (+ 201us) client_negotiation.cc:769] Sending connection context
0809 16:24:24.526356 (+ 140us) client_negotiation.cc:240] Negotiation successful
0809 16:24:24.526550 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":102,"threads_started":1}
I20260809 16:24:24.527278 19574 heartbeater.cc:347] Connected to a master server at 127.21.51.190:38301
I20260809 16:24:24.527390 19574 heartbeater.cc:464] Registering TS with master...
I20260809 16:24:24.527599 19574 heartbeater.cc:511] Master 127.21.51.190:38301 requested a full tablet report, sending...
I20260809 16:24:24.528010 18876 ts_manager.cc:194] Registered new tserver with Master: d0c6d9dbe88d4c97b50835a135d64955 (127.21.51.131:46065)
I20260809 16:24:24.528456 18876 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.51.131@KRBTEST.COM'} at 127.21.51.131:56923
I20260809 16:24:24.538475 21710 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 16:24:24.545882 18926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:24.539940 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34762 (local address 127.21.51.190:38301)
0809 16:24:24.540003 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0809 16:24:24.540005 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:24:24.540145 (+ 140us) server_negotiation.cc:407] Connection header received
0809 16:24:24.540178 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:24:24.540179 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:24:24.540189 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:24:24.540243 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:24:24.540857 (+ 614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:24:24.541387 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:24:24.542114 (+ 727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:24:24.542325 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:24:24.542393 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:24:24.543594 (+ 1201us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:24:24.543595 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:24:24.543596 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:24:24.543602 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:24:24.544942 (+ 1340us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:24:24.545248 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:24:24.545249 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:24:24.545249 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:24:24.545303 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:24:24.545461 (+ 158us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:24:24.545462 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:24:24.545462 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:24:24.545596 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:24:24.545674 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0809 16:24:24.545761 (+ 87us) server_negotiation.cc:299] Negotiation successful
0809 16:24:24.545822 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260809 16:24:24.547699 18876 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34762:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 16:24:24.549430 18876 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 16:24:24.555989 19593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:24.551745 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.129:32913 (local address 127.0.0.1:53670)
0809 16:24:24.551943 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0809 16:24:24.551982 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0809 16:24:24.552047 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:24:24.552435 (+ 388us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:24:24.552439 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:24:24.552441 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:24:24.552525 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:24:24.552527 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:24:24.553608 (+ 1081us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:24:24.553608 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:24:24.554843 (+ 1235us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:24:24.554846 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:24:24.555681 (+ 835us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:24:24.555682 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:24:24.555763 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:24:24.555777 (+ 14us) client_negotiation.cc:769] Sending connection context
0809 16:24:24.555856 (+ 79us) client_negotiation.cc:240] Negotiation successful
0809 16:24:24.555909 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":163,"spinlock_wait_cycles":384,"thread_start_us":49,"threads_started":1}
I20260809 16:24:24.556033 19595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:24.551990 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53670 (local address 127.21.51.129:32913)
0809 16:24:24.552283 (+ 293us) server_negotiation.cc:206] Beginning negotiation
0809 16:24:24.552287 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 16:24:24.552296 (+ 9us) server_negotiation.cc:407] Connection header received
0809 16:24:24.552333 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:24:24.552335 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:24:24.552350 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:24:24.552419 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:24:24.552625 (+ 206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:24:24.553511 (+ 886us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:24:24.554987 (+ 1476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:24:24.555594 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:24:24.555681 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:24:24.555795 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0809 16:24:24.555868 (+ 73us) server_negotiation.cc:299] Negotiation successful
0809 16:24:24.555908 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":260,"thread_start_us":80,"threads_started":1}
I20260809 16:24:24.556339 19592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:24.551817 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.130:34939 (local address 127.0.0.1:48506)
0809 16:24:24.552525 (+ 708us) negotiation.cc:107] Waiting for socket to connect
0809 16:24:24.552537 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 16:24:24.552591 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:24:24.552968 (+ 377us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:24:24.552968 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:24:24.552971 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:24:24.553008 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:24:24.553009 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:24:24.554145 (+ 1136us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:24:24.554146 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:24:24.555172 (+ 1026us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:24:24.555175 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:24:24.556120 (+ 945us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:24:24.556121 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:24:24.556171 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:24:24.556173 (+ 2us) client_negotiation.cc:769] Sending connection context
0809 16:24:24.556217 (+ 44us) client_negotiation.cc:240] Negotiation successful
0809 16:24:24.556253 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":694,"thread_start_us":68,"threads_started":1}
I20260809 16:24:24.556546 19594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:24.551769 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.131:46065 (local address 127.0.0.1:59344)
0809 16:24:24.551943 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0809 16:24:24.551981 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0809 16:24:24.552047 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:24:24.552502 (+ 455us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:24:24.552503 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:24:24.552505 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:24:24.552567 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:24:24.552569 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:24:24.553924 (+ 1355us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:24:24.553925 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:24:24.555014 (+ 1089us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:24:24.555016 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:24:24.555962 (+ 946us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:24:24.555962 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:24:24.556068 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:24:24.556072 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 16:24:24.556314 (+ 242us) client_negotiation.cc:240] Negotiation successful
0809 16:24:24.556347 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":96,"threads_started":1}
I20260809 16:24:24.556631 19598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:24.552237 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48506 (local address 127.21.51.130:34939)
0809 16:24:24.552410 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0809 16:24:24.552415 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 16:24:24.552734 (+ 319us) server_negotiation.cc:407] Connection header received
0809 16:24:24.552854 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:24:24.552857 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:24:24.552873 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:24:24.552949 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:24:24.553077 (+ 128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:24:24.554042 (+ 965us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:24:24.555275 (+ 1233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:24:24.555976 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:24:24.556037 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:24:24.556088 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0809 16:24:24.556514 (+ 426us) server_negotiation.cc:299] Negotiation successful
0809 16:24:24.556555 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":139,"thread_start_us":104,"threads_started":1}
I20260809 16:24:24.556668 19596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:24.551997 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59344 (local address 127.21.51.131:46065)
0809 16:24:24.552206 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0809 16:24:24.552211 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 16:24:24.552224 (+ 13us) server_negotiation.cc:407] Connection header received
0809 16:24:24.552266 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:24:24.552268 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:24:24.552285 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:24:24.552376 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:24:24.552654 (+ 278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:24:24.553755 (+ 1101us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:24:24.555089 (+ 1334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:24:24.555797 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:24:24.555869 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:24:24.555921 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0809 16:24:24.556394 (+ 473us) server_negotiation.cc:299] Negotiation successful
0809 16:24:24.556526 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":162,"thread_start_us":89,"threads_started":1}
I20260809 16:24:24.556933 19065 tablet_service.cc:1467] Processing CreateTablet for tablet b21cf1a097734750a54fca8acfbec4eb (DEFAULT_TABLE table=test-table [id=a3319eec9d72484bb2f1374a6a0b2585]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:24:24.557313 19065 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b21cf1a097734750a54fca8acfbec4eb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:24:24.557271 19259 tablet_service.cc:1467] Processing CreateTablet for tablet b21cf1a097734750a54fca8acfbec4eb (DEFAULT_TABLE table=test-table [id=a3319eec9d72484bb2f1374a6a0b2585]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:24:24.557585 19259 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b21cf1a097734750a54fca8acfbec4eb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:24:24.559954 19599 tablet_bootstrap.cc:492] T b21cf1a097734750a54fca8acfbec4eb P 59815c45af0f45f69215e4b9f19eec0f: Bootstrap starting.
I20260809 16:24:24.560349 19603 tablet_bootstrap.cc:492] T b21cf1a097734750a54fca8acfbec4eb P b863f9eba8a94b1282a2e78435824cf9: Bootstrap starting.
I20260809 16:24:24.560607 19468 tablet_service.cc:1467] Processing CreateTablet for tablet b21cf1a097734750a54fca8acfbec4eb (DEFAULT_TABLE table=test-table [id=a3319eec9d72484bb2f1374a6a0b2585]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:24:24.560729 19599 tablet_bootstrap.cc:654] T b21cf1a097734750a54fca8acfbec4eb P 59815c45af0f45f69215e4b9f19eec0f: Neither blocks nor log segments found. Creating new log.
I20260809 16:24:24.560945 19468 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b21cf1a097734750a54fca8acfbec4eb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:24:24.561064 19603 tablet_bootstrap.cc:654] T b21cf1a097734750a54fca8acfbec4eb P b863f9eba8a94b1282a2e78435824cf9: Neither blocks nor log segments found. Creating new log.
I20260809 16:24:24.561357 19599 log.cc:824] T b21cf1a097734750a54fca8acfbec4eb P 59815c45af0f45f69215e4b9f19eec0f: Log is configured to *not* fsync() on all Append() calls
I20260809 16:24:24.561671 19603 log.cc:824] T b21cf1a097734750a54fca8acfbec4eb P b863f9eba8a94b1282a2e78435824cf9: Log is configured to *not* fsync() on all Append() calls
I20260809 16:24:24.562196 19599 tablet_bootstrap.cc:492] T b21cf1a097734750a54fca8acfbec4eb P 59815c45af0f45f69215e4b9f19eec0f: No bootstrap required, opened a new log
I20260809 16:24:24.562273 19599 ts_tablet_manager.cc:1397] T b21cf1a097734750a54fca8acfbec4eb P 59815c45af0f45f69215e4b9f19eec0f: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260809 16:24:24.562451 19603 tablet_bootstrap.cc:492] T b21cf1a097734750a54fca8acfbec4eb P b863f9eba8a94b1282a2e78435824cf9: No bootstrap required, opened a new log
I20260809 16:24:24.562515 19603 ts_tablet_manager.cc:1397] T b21cf1a097734750a54fca8acfbec4eb P b863f9eba8a94b1282a2e78435824cf9: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 16:24:24.564280 19599 raft_consensus.cc:348] T b21cf1a097734750a54fca8acfbec4eb P 59815c45af0f45f69215e4b9f19eec0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59815c45af0f45f69215e4b9f19eec0f" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 32913 } } peers { permanent_uuid: "d0c6d9dbe88d4c97b50835a135d64955" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 46065 } } peers { permanent_uuid: "b863f9eba8a94b1282a2e78435824cf9" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 34939 } }
I20260809 16:24:24.564467 19599 raft_consensus.cc:374] T b21cf1a097734750a54fca8acfbec4eb P 59815c45af0f45f69215e4b9f19eec0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:24:24.564507 19599 raft_consensus.cc:729] T b21cf1a097734750a54fca8acfbec4eb P 59815c45af0f45f69215e4b9f19eec0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 59815c45af0f45f69215e4b9f19eec0f, State: Initialized, Role: FOLLOWER
I20260809 16:24:24.564433 19603 raft_consensus.cc:348] T b21cf1a097734750a54fca8acfbec4eb P b863f9eba8a94b1282a2e78435824cf9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59815c45af0f45f69215e4b9f19eec0f" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 32913 } } peers { permanent_uuid: "d0c6d9dbe88d4c97b50835a135d64955" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 46065 } } peers { permanent_uuid: "b863f9eba8a94b1282a2e78435824cf9" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 34939 } }
I20260809 16:24:24.564591 19603 raft_consensus.cc:374] T b21cf1a097734750a54fca8acfbec4eb P b863f9eba8a94b1282a2e78435824cf9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:24:24.564633 19603 raft_consensus.cc:729] T b21cf1a097734750a54fca8acfbec4eb P b863f9eba8a94b1282a2e78435824cf9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b863f9eba8a94b1282a2e78435824cf9, State: Initialized, Role: FOLLOWER
I20260809 16:24:24.564632 19599 consensus_queue.cc:260] T b21cf1a097734750a54fca8acfbec4eb P 59815c45af0f45f69215e4b9f19eec0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59815c45af0f45f69215e4b9f19eec0f" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 32913 } } peers { permanent_uuid: "d0c6d9dbe88d4c97b50835a135d64955" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 46065 } } peers { permanent_uuid: "b863f9eba8a94b1282a2e78435824cf9" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 34939 } }
I20260809 16:24:24.564826 19603 consensus_queue.cc:260] T b21cf1a097734750a54fca8acfbec4eb P b863f9eba8a94b1282a2e78435824cf9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59815c45af0f45f69215e4b9f19eec0f" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 32913 } } peers { permanent_uuid: "d0c6d9dbe88d4c97b50835a135d64955" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 46065 } } peers { permanent_uuid: "b863f9eba8a94b1282a2e78435824cf9" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 34939 } }
I20260809 16:24:24.564965 19159 heartbeater.cc:503] Master 127.21.51.190:38301 was elected leader, sending a full tablet report...
I20260809 16:24:24.565009 19599 ts_tablet_manager.cc:1428] T b21cf1a097734750a54fca8acfbec4eb P 59815c45af0f45f69215e4b9f19eec0f: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260809 16:24:24.565104 19603 ts_tablet_manager.cc:1428] T b21cf1a097734750a54fca8acfbec4eb P b863f9eba8a94b1282a2e78435824cf9: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 16:24:24.565196 19606 tablet_bootstrap.cc:492] T b21cf1a097734750a54fca8acfbec4eb P d0c6d9dbe88d4c97b50835a135d64955: Bootstrap starting.
I20260809 16:24:24.565279 19370 heartbeater.cc:503] Master 127.21.51.190:38301 was elected leader, sending a full tablet report...
I20260809 16:24:24.565857 19606 tablet_bootstrap.cc:654] T b21cf1a097734750a54fca8acfbec4eb P d0c6d9dbe88d4c97b50835a135d64955: Neither blocks nor log segments found. Creating new log.
I20260809 16:24:24.566393 19606 log.cc:824] T b21cf1a097734750a54fca8acfbec4eb P d0c6d9dbe88d4c97b50835a135d64955: Log is configured to *not* fsync() on all Append() calls
I20260809 16:24:24.567055 19606 tablet_bootstrap.cc:492] T b21cf1a097734750a54fca8acfbec4eb P d0c6d9dbe88d4c97b50835a135d64955: No bootstrap required, opened a new log
I20260809 16:24:24.567108 19606 ts_tablet_manager.cc:1397] T b21cf1a097734750a54fca8acfbec4eb P d0c6d9dbe88d4c97b50835a135d64955: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 16:24:24.568317 19606 raft_consensus.cc:348] T b21cf1a097734750a54fca8acfbec4eb P d0c6d9dbe88d4c97b50835a135d64955 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59815c45af0f45f69215e4b9f19eec0f" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 32913 } } peers { permanent_uuid: "d0c6d9dbe88d4c97b50835a135d64955" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 46065 } } peers { permanent_uuid: "b863f9eba8a94b1282a2e78435824cf9" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 34939 } }
I20260809 16:24:24.568446 19606 raft_consensus.cc:374] T b21cf1a097734750a54fca8acfbec4eb P d0c6d9dbe88d4c97b50835a135d64955 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:24:24.568468 19606 raft_consensus.cc:729] T b21cf1a097734750a54fca8acfbec4eb P d0c6d9dbe88d4c97b50835a135d64955 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d0c6d9dbe88d4c97b50835a135d64955, State: Initialized, Role: FOLLOWER
I20260809 16:24:24.568548 19606 consensus_queue.cc:260] T b21cf1a097734750a54fca8acfbec4eb P d0c6d9dbe88d4c97b50835a135d64955 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59815c45af0f45f69215e4b9f19eec0f" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 32913 } } peers { permanent_uuid: "d0c6d9dbe88d4c97b50835a135d64955" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 46065 } } peers { permanent_uuid: "b863f9eba8a94b1282a2e78435824cf9" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 34939 } }
I20260809 16:24:24.568804 19606 ts_tablet_manager.cc:1428] T b21cf1a097734750a54fca8acfbec4eb P d0c6d9dbe88d4c97b50835a135d64955: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260809 16:24:24.568817 19574 heartbeater.cc:503] Master 127.21.51.190:38301 was elected leader, sending a full tablet report...
I20260809 16:24:24.604045 19607 raft_consensus.cc:482] T b21cf1a097734750a54fca8acfbec4eb P 59815c45af0f45f69215e4b9f19eec0f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 16:24:24.604211 19607 raft_consensus.cc:504] T b21cf1a097734750a54fca8acfbec4eb P 59815c45af0f45f69215e4b9f19eec0f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59815c45af0f45f69215e4b9f19eec0f" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 32913 } } peers { permanent_uuid: "d0c6d9dbe88d4c97b50835a135d64955" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 46065 } } peers { permanent_uuid: "b863f9eba8a94b1282a2e78435824cf9" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 34939 } }
I20260809 16:24:24.604600 19607 leader_election.cc:288] T b21cf1a097734750a54fca8acfbec4eb P 59815c45af0f45f69215e4b9f19eec0f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d0c6d9dbe88d4c97b50835a135d64955 (127.21.51.131:46065), b863f9eba8a94b1282a2e78435824cf9 (127.21.51.130:34939)
I20260809 16:24:24.607677 19596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:24.604922 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.51.129:53935 (local address 127.21.51.131:46065)
0809 16:24:24.605001 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0809 16:24:24.605003 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:24:24.605011 (+ 8us) server_negotiation.cc:407] Connection header received
0809 16:24:24.605044 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:24:24.605045 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:24:24.605052 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:24:24.605127 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:24:24.605351 (+ 224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:24:24.605878 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:24:24.606863 (+ 985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:24:24.607305 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:24:24.607361 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:24:24.607369 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0809 16:24:24.607560 (+ 191us) server_negotiation.cc:299] Negotiation successful
0809 16:24:24.607606 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260809 16:24:24.607677 19162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:24.604806 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.131:46065 (local address 127.21.51.129:53935)
0809 16:24:24.604865 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0809 16:24:24.604876 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 16:24:24.604920 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:24:24.605149 (+ 229us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:24:24.605150 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:24:24.605152 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:24:24.605232 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:24:24.605234 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:24:24.605958 (+ 724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:24:24.605959 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:24:24.606771 (+ 812us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:24:24.606773 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:24:24.607392 (+ 619us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:24:24.607393 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:24:24.607484 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:24:24.607489 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 16:24:24.607541 (+ 52us) client_negotiation.cc:240] Negotiation successful
0809 16:24:24.607586 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39}
I20260809 16:24:24.607762 19598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:24.604891 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.51.129:34285 (local address 127.21.51.130:34939)
0809 16:24:24.604941 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0809 16:24:24.604943 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:24:24.605158 (+ 215us) server_negotiation.cc:407] Connection header received
0809 16:24:24.605222 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:24:24.605224 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:24:24.605234 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:24:24.605284 (+ 50us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:24:24.605449 (+ 165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:24:24.606036 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:24:24.606981 (+ 945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:24:24.607388 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:24:24.607465 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:24:24.607473 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0809 16:24:24.607646 (+ 173us) server_negotiation.cc:299] Negotiation successful
0809 16:24:24.607688 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260809 16:24:24.607796 19612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:24.604806 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.130:34939 (local address 127.21.51.129:34285)
0809 16:24:24.605100 (+ 294us) negotiation.cc:107] Waiting for socket to connect
0809 16:24:24.605108 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0809 16:24:24.605160 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:24:24.605287 (+ 127us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:24:24.605287 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:24:24.605289 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:24:24.605365 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:24:24.605367 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:24:24.606123 (+ 756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:24:24.606124 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:24:24.606889 (+ 765us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:24:24.606891 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:24:24.607479 (+ 588us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:24:24.607480 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:24:24.607558 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:24:24.607563 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 16:24:24.607620 (+ 57us) client_negotiation.cc:240] Negotiation successful
0809 16:24:24.607658 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":277,"mutex_wait_us":86,"thread_start_us":152,"threads_started":1}
I20260809 16:24:24.608075 19504 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b21cf1a097734750a54fca8acfbec4eb" candidate_uuid: "59815c45af0f45f69215e4b9f19eec0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d0c6d9dbe88d4c97b50835a135d64955" is_pre_election: true
I20260809 16:24:24.608156 19291 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b21cf1a097734750a54fca8acfbec4eb" candidate_uuid: "59815c45af0f45f69215e4b9f19eec0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b863f9eba8a94b1282a2e78435824cf9" is_pre_election: true
I20260809 16:24:24.608237 19504 raft_consensus.cc:2454] T b21cf1a097734750a54fca8acfbec4eb P d0c6d9dbe88d4c97b50835a135d64955 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 59815c45af0f45f69215e4b9f19eec0f in term 0.
I20260809 16:24:24.608346 19291 raft_consensus.cc:2454] T b21cf1a097734750a54fca8acfbec4eb P b863f9eba8a94b1282a2e78435824cf9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 59815c45af0f45f69215e4b9f19eec0f in term 0.
I20260809 16:24:24.608445 18994 leader_election.cc:302] T b21cf1a097734750a54fca8acfbec4eb P 59815c45af0f45f69215e4b9f19eec0f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 59815c45af0f45f69215e4b9f19eec0f, d0c6d9dbe88d4c97b50835a135d64955; no voters:
I20260809 16:24:24.608599 19607 raft_consensus.cc:2781] T b21cf1a097734750a54fca8acfbec4eb P 59815c45af0f45f69215e4b9f19eec0f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 16:24:24.608665 19607 raft_consensus.cc:482] T b21cf1a097734750a54fca8acfbec4eb P 59815c45af0f45f69215e4b9f19eec0f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 16:24:24.608748 19607 raft_consensus.cc:3037] T b21cf1a097734750a54fca8acfbec4eb P 59815c45af0f45f69215e4b9f19eec0f [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:24:24.609479 19607 raft_consensus.cc:504] T b21cf1a097734750a54fca8acfbec4eb P 59815c45af0f45f69215e4b9f19eec0f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59815c45af0f45f69215e4b9f19eec0f" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 32913 } } peers { permanent_uuid: "d0c6d9dbe88d4c97b50835a135d64955" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 46065 } } peers { permanent_uuid: "b863f9eba8a94b1282a2e78435824cf9" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 34939 } }
I20260809 16:24:24.609627 19607 leader_election.cc:288] T b21cf1a097734750a54fca8acfbec4eb P 59815c45af0f45f69215e4b9f19eec0f [CANDIDATE]: Term 1 election: Requested vote from peers d0c6d9dbe88d4c97b50835a135d64955 (127.21.51.131:46065), b863f9eba8a94b1282a2e78435824cf9 (127.21.51.130:34939)
I20260809 16:24:24.609781 19291 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b21cf1a097734750a54fca8acfbec4eb" candidate_uuid: "59815c45af0f45f69215e4b9f19eec0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b863f9eba8a94b1282a2e78435824cf9"
I20260809 16:24:24.609808 19504 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b21cf1a097734750a54fca8acfbec4eb" candidate_uuid: "59815c45af0f45f69215e4b9f19eec0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d0c6d9dbe88d4c97b50835a135d64955"
I20260809 16:24:24.609882 19291 raft_consensus.cc:3037] T b21cf1a097734750a54fca8acfbec4eb P b863f9eba8a94b1282a2e78435824cf9 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:24:24.609877 19504 raft_consensus.cc:3037] T b21cf1a097734750a54fca8acfbec4eb P d0c6d9dbe88d4c97b50835a135d64955 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:24:24.610582 19291 raft_consensus.cc:2454] T b21cf1a097734750a54fca8acfbec4eb P b863f9eba8a94b1282a2e78435824cf9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 59815c45af0f45f69215e4b9f19eec0f in term 1.
I20260809 16:24:24.610607 19504 raft_consensus.cc:2454] T b21cf1a097734750a54fca8acfbec4eb P d0c6d9dbe88d4c97b50835a135d64955 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 59815c45af0f45f69215e4b9f19eec0f in term 1.
I20260809 16:24:24.610790 18997 leader_election.cc:302] T b21cf1a097734750a54fca8acfbec4eb P 59815c45af0f45f69215e4b9f19eec0f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 59815c45af0f45f69215e4b9f19eec0f, b863f9eba8a94b1282a2e78435824cf9; no voters:
I20260809 16:24:24.610894 19607 raft_consensus.cc:2781] T b21cf1a097734750a54fca8acfbec4eb P 59815c45af0f45f69215e4b9f19eec0f [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:24:24.611057 19607 raft_consensus.cc:686] T b21cf1a097734750a54fca8acfbec4eb P 59815c45af0f45f69215e4b9f19eec0f [term 1 LEADER]: Becoming Leader. State: Replica: 59815c45af0f45f69215e4b9f19eec0f, State: Running, Role: LEADER
I20260809 16:24:24.611156 19607 consensus_queue.cc:237] T b21cf1a097734750a54fca8acfbec4eb P 59815c45af0f45f69215e4b9f19eec0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59815c45af0f45f69215e4b9f19eec0f" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 32913 } } peers { permanent_uuid: "d0c6d9dbe88d4c97b50835a135d64955" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 46065 } } peers { permanent_uuid: "b863f9eba8a94b1282a2e78435824cf9" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 34939 } }
I20260809 16:24:24.612030 18876 catalog_manager.cc:5495] T b21cf1a097734750a54fca8acfbec4eb P 59815c45af0f45f69215e4b9f19eec0f reported cstate change: term changed from 0 to 1, leader changed from <none> to 59815c45af0f45f69215e4b9f19eec0f (127.21.51.129). New cstate: current_term: 1 leader_uuid: "59815c45af0f45f69215e4b9f19eec0f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59815c45af0f45f69215e4b9f19eec0f" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 32913 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d0c6d9dbe88d4c97b50835a135d64955" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 46065 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b863f9eba8a94b1282a2e78435824cf9" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 34939 } health_report { overall_health: UNKNOWN } } }
W20260809 16:24:24.621791 19372 tablet.cc:2369] T b21cf1a097734750a54fca8acfbec4eb P b863f9eba8a94b1282a2e78435824cf9: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 16:24:24.630506 19595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:24.628037 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53684 (local address 127.21.51.129:32913)
0809 16:24:24.628103 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0809 16:24:24.628105 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:24:24.628114 (+ 9us) server_negotiation.cc:407] Connection header received
0809 16:24:24.628165 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:24:24.628166 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:24:24.628183 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:24:24.628264 (+ 81us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 16:24:24.628400 (+ 136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:24:24.628933 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:24:24.629704 (+ 771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:24:24.629901 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:24:24.629979 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:24:24.630155 (+ 176us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 16:24:24.630256 (+ 101us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 16:24:24.630346 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0809 16:24:24.630394 (+ 48us) server_negotiation.cc:299] Negotiation successful
0809 16:24:24.630415 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260809 16:24:24.632439 19504 raft_consensus.cc:1261] T b21cf1a097734750a54fca8acfbec4eb P d0c6d9dbe88d4c97b50835a135d64955 [term 1 FOLLOWER]: Refusing update from remote peer 59815c45af0f45f69215e4b9f19eec0f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 16:24:24.632504 19291 raft_consensus.cc:1261] T b21cf1a097734750a54fca8acfbec4eb P b863f9eba8a94b1282a2e78435824cf9 [term 1 FOLLOWER]: Refusing update from remote peer 59815c45af0f45f69215e4b9f19eec0f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 16:24:24.632820 19607 consensus_queue.cc:1035] T b21cf1a097734750a54fca8acfbec4eb P 59815c45af0f45f69215e4b9f19eec0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d0c6d9dbe88d4c97b50835a135d64955" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 46065 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 16:24:24.632987 19613 consensus_queue.cc:1035] T b21cf1a097734750a54fca8acfbec4eb P 59815c45af0f45f69215e4b9f19eec0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "b863f9eba8a94b1282a2e78435824cf9" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 34939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 16:24:24.635783 19619 mvcc.cc:205] Tried to move back new op lower bound from 7316654754331406336 to 7316654754247864320. Current Snapshot: MvccSnapshot[applied={T|T < 7316654754331406336}]
I20260809 16:24:24.638015 18878 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34762:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 16:24:24.638131 18878 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34762:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 16:24:24.638988 18878 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 355e97e0fa674a6e84e7155fb4618297: Sending DeleteTablet for 3 replicas of tablet b21cf1a097734750a54fca8acfbec4eb
I20260809 16:24:24.639366 19468 tablet_service.cc:1514] Processing DeleteTablet for tablet b21cf1a097734750a54fca8acfbec4eb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:24:24 UTC) from {username='kudu', principal='kudu/127.21.51.190@KRBTEST.COM'} at 127.0.0.1:59344
I20260809 16:24:24.639609 19636 tablet_replica.cc:330] T b21cf1a097734750a54fca8acfbec4eb P d0c6d9dbe88d4c97b50835a135d64955: stopping tablet replica
I20260809 16:24:24.639614 19259 tablet_service.cc:1514] Processing DeleteTablet for tablet b21cf1a097734750a54fca8acfbec4eb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:24:24 UTC) from {username='kudu', principal='kudu/127.21.51.190@KRBTEST.COM'} at 127.0.0.1:48506
I20260809 16:24:24.639729 19636 raft_consensus.cc:2229] T b21cf1a097734750a54fca8acfbec4eb P d0c6d9dbe88d4c97b50835a135d64955 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 16:24:24.639834 19636 raft_consensus.cc:2258] T b21cf1a097734750a54fca8acfbec4eb P d0c6d9dbe88d4c97b50835a135d64955 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 16:24:24.639986 19065 tablet_service.cc:1514] Processing DeleteTablet for tablet b21cf1a097734750a54fca8acfbec4eb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:24:24 UTC) from {username='kudu', principal='kudu/127.21.51.190@KRBTEST.COM'} at 127.0.0.1:53670
I20260809 16:24:24.640146 21710 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrNqUCn/build/release/bin/kudu with pid 18958
I20260809 16:24:24.640235 19638 tablet_replica.cc:330] T b21cf1a097734750a54fca8acfbec4eb P 59815c45af0f45f69215e4b9f19eec0f: stopping tablet replica
I20260809 16:24:24.640321 19638 raft_consensus.cc:2229] T b21cf1a097734750a54fca8acfbec4eb P 59815c45af0f45f69215e4b9f19eec0f [term 1 LEADER]: Raft consensus shutting down.
I20260809 16:24:24.640409 19638 raft_consensus.cc:2258] T b21cf1a097734750a54fca8acfbec4eb P 59815c45af0f45f69215e4b9f19eec0f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 16:24:24.649909 19637 tablet_replica.cc:330] T b21cf1a097734750a54fca8acfbec4eb P b863f9eba8a94b1282a2e78435824cf9: stopping tablet replica
I20260809 16:24:24.650072 19637 raft_consensus.cc:2229] T b21cf1a097734750a54fca8acfbec4eb P b863f9eba8a94b1282a2e78435824cf9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 16:24:24.650202 19637 raft_consensus.cc:2258] T b21cf1a097734750a54fca8acfbec4eb P b863f9eba8a94b1282a2e78435824cf9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 16:24:24.650344 19636 ts_tablet_manager.cc:1905] T b21cf1a097734750a54fca8acfbec4eb P d0c6d9dbe88d4c97b50835a135d64955: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 16:24:24.650683 19637 ts_tablet_manager.cc:1905] T b21cf1a097734750a54fca8acfbec4eb P b863f9eba8a94b1282a2e78435824cf9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 16:24:24.652267 19637 ts_tablet_manager.cc:1918] T b21cf1a097734750a54fca8acfbec4eb P b863f9eba8a94b1282a2e78435824cf9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 16:24:24.652357 19637 log.cc:1196] T b21cf1a097734750a54fca8acfbec4eb P b863f9eba8a94b1282a2e78435824cf9: Deleting WAL directory at /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786292417722232-21710-0/minicluster-data/ts-1/wal/wals/b21cf1a097734750a54fca8acfbec4eb
I20260809 16:24:24.652412 19636 ts_tablet_manager.cc:1918] T b21cf1a097734750a54fca8acfbec4eb P d0c6d9dbe88d4c97b50835a135d64955: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 16:24:24.652494 19636 log.cc:1196] T b21cf1a097734750a54fca8acfbec4eb P d0c6d9dbe88d4c97b50835a135d64955: Deleting WAL directory at /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786292417722232-21710-0/minicluster-data/ts-2/wal/wals/b21cf1a097734750a54fca8acfbec4eb
I20260809 16:24:24.652661 19637 ts_tablet_manager.cc:1939] T b21cf1a097734750a54fca8acfbec4eb P b863f9eba8a94b1282a2e78435824cf9: Deleting consensus metadata
I20260809 16:24:24.652829 19636 ts_tablet_manager.cc:1939] T b21cf1a097734750a54fca8acfbec4eb P d0c6d9dbe88d4c97b50835a135d64955: Deleting consensus metadata
I20260809 16:24:24.653194 18857 catalog_manager.cc:4841] TS b863f9eba8a94b1282a2e78435824cf9 (127.21.51.130:34939): tablet b21cf1a097734750a54fca8acfbec4eb (table test-table [id=a3319eec9d72484bb2f1374a6a0b2585]) successfully deleted
I20260809 16:24:24.653249 18853 catalog_manager.cc:4841] TS d0c6d9dbe88d4c97b50835a135d64955 (127.21.51.131:46065): tablet b21cf1a097734750a54fca8acfbec4eb (table test-table [id=a3319eec9d72484bb2f1374a6a0b2585]) successfully deleted
W20260809 16:24:24.655069 18856 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.21.51.129:32913 (error 108)
I20260809 16:24:24.655369 21710 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrNqUCn/build/release/bin/kudu with pid 19166
I20260809 16:24:24.657150 19594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:24.655468 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.129:32913 (local address 127.0.0.1:53686)
0809 16:24:24.657064 (+ 1596us) negotiation.cc:107] Waiting for socket to connect
0809 16:24:24.657110 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.51.129:32913: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1576}
W20260809 16:24:24.659582 18856 catalog_manager.cc:4584] TS 59815c45af0f45f69215e4b9f19eec0f (127.21.51.129:32913): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet b21cf1a097734750a54fca8acfbec4eb: Network error: Client connection negotiation failed: client connection to 127.21.51.129:32913: connect: Connection refused (error 111)
I20260809 16:24:24.673857 21710 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrNqUCn/build/release/bin/kudu with pid 19378
I20260809 16:24:24.685164 21710 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrNqUCn/build/release/bin/kudu with pid 18826
2026-08-09T16:24:24Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (2197 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-08-09T16:24:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T16:24:24Z Disabled control of system clock
I20260809 16:24:24.721141 21710 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrNqUCn/build/release/bin/kudu
/tmp/dist-test-taskrNqUCn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786292417722232-21710-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786292417722232-21710-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786292417722232-21710-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786292417722232-21710-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.51.190:34021
--webserver_interface=127.21.51.190
--webserver_port=0
--builtin_ntp_servers=127.21.51.148:45015
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.51.190:34021
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 16:24:24.812656 19645 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:24:24.812958 19645 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:24:24.812980 19645 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:24:24.814167 19645 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 16:24:24.814206 19645 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:24:24.814220 19645 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:24:24.814241 19645 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 16:24:24.814257 19645 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 16:24:24.815775 19645 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.51.148:45015
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786292417722232-21710-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786292417722232-21710-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.51.190:34021
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.51.190:34021
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786292417722232-21710-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.51.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19645
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786292417722232-21710-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:24:24.815990 19645 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:24:24.816256 19645 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:24:24.821280 19662 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:24:24.821324 19653 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:24:24.821357 19654 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:24:24.821499 19645 server_base.cc:1034] running on GCE node
I20260809 16:24:24.821830 19645 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:24:24.822146 19645 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:24:24.823326 19645 hybrid_clock.cc:648] HybridClock initialized: now 1786292664823301 us; error 38 us; skew 500 ppm
I20260809 16:24:24.824632 19645 webserver.cc:460] Webserver started at http://127.21.51.190:34415/ using document root <none> and password file <none>
I20260809 16:24:24.824940 19645 fs_manager.cc:359] Metadata directory not provided
I20260809 16:24:24.825040 19645 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:24:24.825217 19645 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:24:24.826311 19645 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786292417722232-21710-0/minicluster-data/master-0/data/instance:
uuid: "603e269124fe45c88d9278d0cdc2e5f1"
format_stamp: "Formatted at 2026-08-09 16:24:24 on dist-test-slave-s11t"
server_key: "be42197f9f08d114ccacdb2f62ec0785"
server_key_iv: "92e6cfbd9b2b952296fb77001b502725"
server_key_version: "encryptionkey@0"
I20260809 16:24:24.826683 19645 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786292417722232-21710-0/minicluster-data/master-0/wal/instance:
uuid: "603e269124fe45c88d9278d0cdc2e5f1"
format_stamp: "Formatted at 2026-08-09 16:24:24 on dist-test-slave-s11t"
server_key: "be42197f9f08d114ccacdb2f62ec0785"
server_key_iv: "92e6cfbd9b2b952296fb77001b502725"
server_key_version: "encryptionkey@0"
I20260809 16:24:24.828081 19645 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 16:24:24.828989 19668 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:24:24.829178 19645 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:24:24.829304 19645 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786292417722232-21710-0/minicluster-data/master-0/data,/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786292417722232-21710-0/minicluster-data/master-0/wal
uuid: "603e269124fe45c88d9278d0cdc2e5f1"
format_stamp: "Formatted at 2026-08-09 16:24:24 on dist-test-slave-s11t"
server_key: "be42197f9f08d114ccacdb2f62ec0785"
server_key_iv: "92e6cfbd9b2b952296fb77001b502725"
server_key_version: "encryptionkey@0"
I20260809 16:24:24.829381 19645 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786292417722232-21710-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786292417722232-21710-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786292417722232-21710-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:24:24.841089 19645 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:24:24.841393 19645 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:24:24.841547 19645 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:24:24.845538 19645 rpc_server.cc:307] RPC server started. Bound to: 127.21.51.190:34021
I20260809 16:24:24.845572 19746 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.51.190:34021 every 8 connection(s)
I20260809 16:24:24.845923 19645 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786292417722232-21710-0/minicluster-data/master-0/data/info.pb
I20260809 16:24:24.846866 19750 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:24:24.849184 19750 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 603e269124fe45c88d9278d0cdc2e5f1: Bootstrap starting.
I20260809 16:24:24.849756 19750 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 603e269124fe45c88d9278d0cdc2e5f1: Neither blocks nor log segments found. Creating new log.
I20260809 16:24:24.850067 19750 log.cc:824] T 00000000000000000000000000000000 P 603e269124fe45c88d9278d0cdc2e5f1: Log is configured to *not* fsync() on all Append() calls
I20260809 16:24:24.850730 19750 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 603e269124fe45c88d9278d0cdc2e5f1: No bootstrap required, opened a new log
I20260809 16:24:24.852247 19750 raft_consensus.cc:348] T 00000000000000000000000000000000 P 603e269124fe45c88d9278d0cdc2e5f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "603e269124fe45c88d9278d0cdc2e5f1" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 34021 } }
I20260809 16:24:24.852388 19750 raft_consensus.cc:374] T 00000000000000000000000000000000 P 603e269124fe45c88d9278d0cdc2e5f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:24:24.852411 19750 raft_consensus.cc:729] T 00000000000000000000000000000000 P 603e269124fe45c88d9278d0cdc2e5f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 603e269124fe45c88d9278d0cdc2e5f1, State: Initialized, Role: FOLLOWER
I20260809 16:24:24.852540 19750 consensus_queue.cc:260] T 00000000000000000000000000000000 P 603e269124fe45c88d9278d0cdc2e5f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "603e269124fe45c88d9278d0cdc2e5f1" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 34021 } }
I20260809 16:24:24.852618 19750 raft_consensus.cc:388] T 00000000000000000000000000000000 P 603e269124fe45c88d9278d0cdc2e5f1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 16:24:24.852640 19750 raft_consensus.cc:482] T 00000000000000000000000000000000 P 603e269124fe45c88d9278d0cdc2e5f1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 16:24:24.852669 19750 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 603e269124fe45c88d9278d0cdc2e5f1 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:24:24.853199 19750 raft_consensus.cc:504] T 00000000000000000000000000000000 P 603e269124fe45c88d9278d0cdc2e5f1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "603e269124fe45c88d9278d0cdc2e5f1" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 34021 } }
I20260809 16:24:24.853283 19750 leader_election.cc:302] T 00000000000000000000000000000000 P 603e269124fe45c88d9278d0cdc2e5f1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 603e269124fe45c88d9278d0cdc2e5f1; no voters:
I20260809 16:24:24.853423 19750 leader_election.cc:288] T 00000000000000000000000000000000 P 603e269124fe45c88d9278d0cdc2e5f1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 16:24:24.853482 19753 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 603e269124fe45c88d9278d0cdc2e5f1 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:24:24.853663 19753 raft_consensus.cc:686] T 00000000000000000000000000000000 P 603e269124fe45c88d9278d0cdc2e5f1 [term 1 LEADER]: Becoming Leader. State: Replica: 603e269124fe45c88d9278d0cdc2e5f1, State: Running, Role: LEADER
I20260809 16:24:24.853734 19750 sys_catalog.cc:564] T 00000000000000000000000000000000 P 603e269124fe45c88d9278d0cdc2e5f1 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 16:24:24.853806 19753 consensus_queue.cc:237] T 00000000000000000000000000000000 P 603e269124fe45c88d9278d0cdc2e5f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "603e269124fe45c88d9278d0cdc2e5f1" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 34021 } }
I20260809 16:24:24.854200 19758 sys_catalog.cc:455] T 00000000000000000000000000000000 P 603e269124fe45c88d9278d0cdc2e5f1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 603e269124fe45c88d9278d0cdc2e5f1. Latest consensus state: current_term: 1 leader_uuid: "603e269124fe45c88d9278d0cdc2e5f1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "603e269124fe45c88d9278d0cdc2e5f1" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 34021 } } }
I20260809 16:24:24.854300 19758 sys_catalog.cc:458] T 00000000000000000000000000000000 P 603e269124fe45c88d9278d0cdc2e5f1 [sys.catalog]: This master's current role is: LEADER
I20260809 16:24:24.854333 19756 sys_catalog.cc:455] T 00000000000000000000000000000000 P 603e269124fe45c88d9278d0cdc2e5f1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "603e269124fe45c88d9278d0cdc2e5f1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "603e269124fe45c88d9278d0cdc2e5f1" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 34021 } } }
I20260809 16:24:24.854476 19756 sys_catalog.cc:458] T 00000000000000000000000000000000 P 603e269124fe45c88d9278d0cdc2e5f1 [sys.catalog]: This master's current role is: LEADER
I20260809 16:24:24.854633 19765 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 16:24:24.855188 19765 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 16:24:24.856206 21710 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrNqUCn/build/release/bin/kudu as pid 19645
I20260809 16:24:24.856321 21710 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786292417722232-21710-0/minicluster-data/master-0/wal/instance
I20260809 16:24:24.856549 21710 external_mini_cluster.cc:1428] Setting key 94683355b522fb3ee686f10548c62daf
I20260809 16:24:24.856961 19765 catalog_manager.cc:1347] Generated new cluster ID: 315f475be20d46959d7d5320558f0ec7
I20260809 16:24:24.857024 19765 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 16:24:24.862871 19781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:24.857261 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38038 (local address 127.21.51.190:34021)
0809 16:24:24.857444 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0809 16:24:24.857452 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 16:24:24.857481 (+ 29us) server_negotiation.cc:407] Connection header received
0809 16:24:24.857585 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:24:24.857591 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:24:24.857663 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:24:24.857791 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:24:24.858069 (+ 278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:24:24.859428 (+ 1359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:24:24.860550 (+ 1122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:24:24.861178 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:24:24.861290 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:24:24.862045 (+ 755us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:24:24.862049 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:24:24.862049 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 16:24:24.862269 (+ 220us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:24:24.862311 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 16:24:24.862324 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:24:24.862385 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0809 16:24:24.862487 (+ 102us) server_negotiation.cc:299] Negotiation successful
0809 16:24:24.862663 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":121,"thread_start_us":75,"threads_started":1}
I20260809 16:24:24.868382 19765 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 16:24:24.868999 19765 catalog_manager.cc:1504] Loading token signing keys...
I20260809 16:24:24.872295 19765 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 603e269124fe45c88d9278d0cdc2e5f1: Generated new TSK 0
I20260809 16:24:24.872466 19765 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 16:24:24.878350 21710 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrNqUCn/build/release/bin/kudu
/tmp/dist-test-taskrNqUCn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786292417722232-21710-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786292417722232-21710-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786292417722232-21710-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786292417722232-21710-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.51.129:0
--local_ip_for_outbound_sockets=127.21.51.129
--webserver_interface=127.21.51.129
--webserver_port=0
--tserver_master_addrs=127.21.51.190:34021
--builtin_ntp_servers=127.21.51.148:45015
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 16:24:24.961791 19782 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:24:24.962016 19782 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:24:24.962052 19782 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:24:24.963177 19782 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:24:24.963241 19782 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:24:24.963284 19782 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.51.129
I20260809 16:24:24.964771 19782 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.51.148:45015
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786292417722232-21710-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786292417722232-21710-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.51.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786292417722232-21710-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.51.129
--webserver_port=0
--tserver_master_addrs=127.21.51.190:34021
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19782
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.51.129
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786292417722232-21710-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:24:24.965031 19782 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:24:24.965337 19782 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:24:24.971511 19791 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:24:24.971511 19789 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:24:24.971849 19782 server_base.cc:1034] running on GCE node
W20260809 16:24:24.971529 19787 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:24:24.972173 19782 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:24:24.972462 19782 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:24:24.973644 19782 hybrid_clock.cc:648] HybridClock initialized: now 1786292664973617 us; error 28 us; skew 500 ppm
I20260809 16:24:24.975033 19782 webserver.cc:460] Webserver started at http://127.21.51.129:38409/ using document root <none> and password file <none>
I20260809 16:24:24.975327 19782 fs_manager.cc:359] Metadata directory not provided
I20260809 16:24:24.975380 19782 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:24:24.975560 19782 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:24:24.976649 19782 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786292417722232-21710-0/minicluster-data/ts-0/data/instance:
uuid: "3f31b397033a4e7baea6521846e46690"
format_stamp: "Formatted at 2026-08-09 16:24:24 on dist-test-slave-s11t"
server_key: "8d7235f069910138a54aa9bd11492d0e"
server_key_iv: "0ee2e1b07158b427bec2cedcdfdef4cb"
server_key_version: "encryptionkey@0"
I20260809 16:24:24.977093 19782 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786292417722232-21710-0/minicluster-data/ts-0/wal/instance:
uuid: "3f31b397033a4e7baea6521846e46690"
format_stamp: "Formatted at 2026-08-09 16:24:24 on dist-test-slave-s11t"
server_key: "8d7235f069910138a54aa9bd11492d0e"
server_key_iv: "0ee2e1b07158b427bec2cedcdfdef4cb"
server_key_version: "encryptionkey@0"
I20260809 16:24:24.978446 19782 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260809 16:24:24.979247 19802 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:24:24.979470 19782 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 16:24:24.979564 19782 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786292417722232-21710-0/minicluster-data/ts-0/data,/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786292417722232-21710-0/minicluster-data/ts-0/wal
uuid: "3f31b397033a4e7baea6521846e46690"
format_stamp: "Formatted at 2026-08-09 16:24:24 on dist-test-slave-s11t"
server_key: "8d7235f069910138a54aa9bd11492d0e"
server_key_iv: "0ee2e1b07158b427bec2cedcdfdef4cb"
server_key_version: "encryptionkey@0"
I20260809 16:24:24.979655 19782 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786292417722232-21710-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786292417722232-21710-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786292417722232-21710-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:24:24.993088 19782 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:24:24.993394 19782 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:24:24.993552 19782 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:24:24.993780 19782 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:24:24.994174 19782 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:24:24.994231 19782 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:24:24.994268 19782 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:24:24.994295 19782 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:24:24.999768 19782 rpc_server.cc:307] RPC server started. Bound to: 127.21.51.129:33047
I20260809 16:24:24.999845 19966 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.51.129:33047 every 8 connection(s)
I20260809 16:24:25.000190 19782 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786292417722232-21710-0/minicluster-data/ts-0/data/info.pb
I20260809 16:24:25.004242 21710 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrNqUCn/build/release/bin/kudu as pid 19782
I20260809 16:24:25.004388 21710 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786292417722232-21710-0/minicluster-data/ts-0/wal/instance
I20260809 16:24:25.004611 21710 external_mini_cluster.cc:1428] Setting key a7581fda43bb2b128f6083973b630724
I20260809 16:24:25.005757 21710 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrNqUCn/build/release/bin/kudu
/tmp/dist-test-taskrNqUCn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786292417722232-21710-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786292417722232-21710-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786292417722232-21710-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786292417722232-21710-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.51.130:0
--local_ip_for_outbound_sockets=127.21.51.130
--webserver_interface=127.21.51.130
--webserver_port=0
--tserver_master_addrs=127.21.51.190:34021
--builtin_ntp_servers=127.21.51.148:45015
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260809 16:24:25.005956 19781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:25.001011 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.51.129:33687 (local address 127.21.51.190:34021)
0809 16:24:25.001059 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0809 16:24:25.001061 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:24:25.001312 (+ 251us) server_negotiation.cc:407] Connection header received
0809 16:24:25.001853 (+ 541us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:24:25.001854 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:24:25.001878 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:24:25.001937 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:24:25.002616 (+ 679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:24:25.003176 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:24:25.004291 (+ 1115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:24:25.004590 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:24:25.004680 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:24:25.005642 (+ 962us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:24:25.005643 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:24:25.005643 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 16:24:25.005648 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:24:25.005667 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 16:24:25.005672 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:24:25.005735 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0809 16:24:25.005802 (+ 67us) server_negotiation.cc:299] Negotiation successful
0809 16:24:25.005835 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260809 16:24:25.006168 19971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:25.000983 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:34021 (local address 127.21.51.129:33687)
0809 16:24:25.001191 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0809 16:24:25.001225 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0809 16:24:25.001746 (+ 521us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:24:25.002005 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:24:25.002014 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:24:25.002022 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:24:25.002519 (+ 497us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:24:25.002524 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:24:25.003281 (+ 757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:24:25.003282 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:24:25.004172 (+ 890us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:24:25.004175 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:24:25.004712 (+ 537us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:24:25.004714 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:24:25.004875 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:24:25.005501 (+ 626us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 16:24:25.005502 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:24:25.005531 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 16:24:25.005554 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:24:25.005738 (+ 184us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:24:25.005741 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:24:25.005743 (+ 2us) client_negotiation.cc:769] Sending connection context
0809 16:24:25.005786 (+ 43us) client_negotiation.cc:240] Negotiation successful
0809 16:24:25.005918 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":95,"threads_started":1}
I20260809 16:24:25.006712 19967 heartbeater.cc:347] Connected to a master server at 127.21.51.190:34021
I20260809 16:24:25.006819 19967 heartbeater.cc:464] Registering TS with master...
I20260809 16:24:25.007005 19967 heartbeater.cc:511] Master 127.21.51.190:34021 requested a full tablet report, sending...
I20260809 16:24:25.007587 19696 ts_manager.cc:194] Registered new tserver with Master: 3f31b397033a4e7baea6521846e46690 (127.21.51.129:33047)
I20260809 16:24:25.008229 19696 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.21.51.129:33687
W20260809 16:24:25.090142 19972 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:24:25.090332 19972 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:24:25.090353 19972 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:24:25.091464 19972 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:24:25.091503 19972 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:24:25.091528 19972 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.51.130
I20260809 16:24:25.092960 19972 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.51.148:45015
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786292417722232-21710-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786292417722232-21710-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.51.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786292417722232-21710-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.51.130
--webserver_port=0
--tserver_master_addrs=127.21.51.190:34021
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19972
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.51.130
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786292417722232-21710-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:24:25.093209 19972 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:24:25.093504 19972 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:24:25.098652 19982 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:24:25.098691 19972 server_base.cc:1034] running on GCE node
W20260809 16:24:25.098699 19981 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:24:25.098652 19984 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:24:25.099092 19972 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:24:25.099368 19972 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:24:25.100526 19972 hybrid_clock.cc:648] HybridClock initialized: now 1786292665100503 us; error 38 us; skew 500 ppm
I20260809 16:24:25.101766 19972 webserver.cc:460] Webserver started at http://127.21.51.130:43687/ using document root <none> and password file <none>
I20260809 16:24:25.102029 19972 fs_manager.cc:359] Metadata directory not provided
I20260809 16:24:25.102100 19972 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:24:25.102245 19972 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:24:25.103282 19972 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786292417722232-21710-0/minicluster-data/ts-1/data/instance:
uuid: "242e85ecda274ead8ba6ad25046e8bc8"
format_stamp: "Formatted at 2026-08-09 16:24:25 on dist-test-slave-s11t"
server_key: "8aab747252518a6e430c1cf344eb9419"
server_key_iv: "01154a6fbf5cc5e892734ba8886f7dc0"
server_key_version: "encryptionkey@0"
I20260809 16:24:25.103655 19972 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786292417722232-21710-0/minicluster-data/ts-1/wal/instance:
uuid: "242e85ecda274ead8ba6ad25046e8bc8"
format_stamp: "Formatted at 2026-08-09 16:24:25 on dist-test-slave-s11t"
server_key: "8aab747252518a6e430c1cf344eb9419"
server_key_iv: "01154a6fbf5cc5e892734ba8886f7dc0"
server_key_version: "encryptionkey@0"
I20260809 16:24:25.105019 19972 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 16:24:25.105731 19995 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:24:25.105911 19972 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 16:24:25.106012 19972 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786292417722232-21710-0/minicluster-data/ts-1/data,/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786292417722232-21710-0/minicluster-data/ts-1/wal
uuid: "242e85ecda274ead8ba6ad25046e8bc8"
format_stamp: "Formatted at 2026-08-09 16:24:25 on dist-test-slave-s11t"
server_key: "8aab747252518a6e430c1cf344eb9419"
server_key_iv: "01154a6fbf5cc5e892734ba8886f7dc0"
server_key_version: "encryptionkey@0"
I20260809 16:24:25.106094 19972 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786292417722232-21710-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786292417722232-21710-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786292417722232-21710-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:24:25.117904 19972 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:24:25.118172 19972 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:24:25.118319 19972 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:24:25.118532 19972 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:24:25.118844 19972 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:24:25.118898 19972 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:24:25.118932 19972 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:24:25.118958 19972 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:24:25.124881 19972 rpc_server.cc:307] RPC server started. Bound to: 127.21.51.130:38349
I20260809 16:24:25.124962 20159 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.51.130:38349 every 8 connection(s)
I20260809 16:24:25.125301 19972 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786292417722232-21710-0/minicluster-data/ts-1/data/info.pb
I20260809 16:24:25.130111 19781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:25.126065 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.51.130:51853 (local address 127.21.51.190:34021)
0809 16:24:25.126117 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0809 16:24:25.126119 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:24:25.126338 (+ 219us) server_negotiation.cc:407] Connection header received
0809 16:24:25.126797 (+ 459us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:24:25.126798 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:24:25.126807 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:24:25.126863 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:24:25.127489 (+ 626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:24:25.127999 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:24:25.128952 (+ 953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:24:25.129134 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:24:25.129187 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:24:25.129899 (+ 712us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:24:25.129900 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:24:25.129900 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 16:24:25.129905 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:24:25.129919 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 16:24:25.129922 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:24:25.129965 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0809 16:24:25.130031 (+ 66us) server_negotiation.cc:299] Negotiation successful
0809 16:24:25.130053 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260809 16:24:25.130357 20162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:25.126021 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:34021 (local address 127.21.51.130:51853)
0809 16:24:25.126209 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0809 16:24:25.126249 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0809 16:24:25.126715 (+ 466us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:24:25.126913 (+ 198us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:24:25.126917 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:24:25.126926 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:24:25.127420 (+ 494us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:24:25.127424 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:24:25.128078 (+ 654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:24:25.128079 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:24:25.128882 (+ 803us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:24:25.128884 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:24:25.129207 (+ 323us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:24:25.129208 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:24:25.129310 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:24:25.129767 (+ 457us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 16:24:25.129768 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:24:25.129795 (+ 27us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 16:24:25.129816 (+ 21us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:24:25.129979 (+ 163us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:24:25.129983 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:24:25.129985 (+ 2us) client_negotiation.cc:769] Sending connection context
0809 16:24:25.130034 (+ 49us) client_negotiation.cc:240] Negotiation successful
0809 16:24:25.130109 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":96,"threads_started":1}
I20260809 16:24:25.130950 20160 heartbeater.cc:347] Connected to a master server at 127.21.51.190:34021
I20260809 16:24:25.131052 20160 heartbeater.cc:464] Registering TS with master...
I20260809 16:24:25.131228 20160 heartbeater.cc:511] Master 127.21.51.190:34021 requested a full tablet report, sending...
I20260809 16:24:25.131651 19696 ts_manager.cc:194] Registered new tserver with Master: 242e85ecda274ead8ba6ad25046e8bc8 (127.21.51.130:38349)
I20260809 16:24:25.131783 21710 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrNqUCn/build/release/bin/kudu as pid 19972
I20260809 16:24:25.131930 21710 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786292417722232-21710-0/minicluster-data/ts-1/wal/instance
I20260809 16:24:25.132113 19696 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.21.51.130:51853
I20260809 16:24:25.132150 21710 external_mini_cluster.cc:1428] Setting key a0815e58787ba044692636d96ec1be33
I20260809 16:24:25.133167 21710 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrNqUCn/build/release/bin/kudu
/tmp/dist-test-taskrNqUCn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786292417722232-21710-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786292417722232-21710-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786292417722232-21710-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786292417722232-21710-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.51.131:0
--local_ip_for_outbound_sockets=127.21.51.131
--webserver_interface=127.21.51.131
--webserver_port=0
--tserver_master_addrs=127.21.51.190:34021
--builtin_ntp_servers=127.21.51.148:45015
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 16:24:25.216384 20163 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:24:25.216580 20163 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:24:25.216601 20163 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:24:25.217751 20163 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:24:25.217789 20163 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:24:25.217813 20163 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.51.131
I20260809 16:24:25.219177 20163 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.51.148:45015
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786292417722232-21710-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786292417722232-21710-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.51.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786292417722232-21710-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.51.131
--webserver_port=0
--tserver_master_addrs=127.21.51.190:34021
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20163
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.51.131
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786292417722232-21710-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:24:25.219427 20163 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:24:25.219719 20163 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:24:25.224944 20171 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:24:25.224972 20177 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:24:25.225147 20163 server_base.cc:1034] running on GCE node
W20260809 16:24:25.224956 20172 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:24:25.225436 20163 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:24:25.225674 20163 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:24:25.226840 20163 hybrid_clock.cc:648] HybridClock initialized: now 1786292665226824 us; error 32 us; skew 500 ppm
I20260809 16:24:25.228166 20163 webserver.cc:460] Webserver started at http://127.21.51.131:44509/ using document root <none> and password file <none>
I20260809 16:24:25.228433 20163 fs_manager.cc:359] Metadata directory not provided
I20260809 16:24:25.228509 20163 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:24:25.228657 20163 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:24:25.229699 20163 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786292417722232-21710-0/minicluster-data/ts-2/data/instance:
uuid: "81d126f4ef9a4713bc29e71874b5a32d"
format_stamp: "Formatted at 2026-08-09 16:24:25 on dist-test-slave-s11t"
server_key: "70b4a428b166f9f2826651a27629b4b2"
server_key_iv: "d1c26f337882deee384bd7a9eb8dfac8"
server_key_version: "encryptionkey@0"
I20260809 16:24:25.230067 20163 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786292417722232-21710-0/minicluster-data/ts-2/wal/instance:
uuid: "81d126f4ef9a4713bc29e71874b5a32d"
format_stamp: "Formatted at 2026-08-09 16:24:25 on dist-test-slave-s11t"
server_key: "70b4a428b166f9f2826651a27629b4b2"
server_key_iv: "d1c26f337882deee384bd7a9eb8dfac8"
server_key_version: "encryptionkey@0"
I20260809 16:24:25.231384 20163 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260809 16:24:25.232151 20184 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:24:25.232326 20163 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 16:24:25.232438 20163 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786292417722232-21710-0/minicluster-data/ts-2/data,/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786292417722232-21710-0/minicluster-data/ts-2/wal
uuid: "81d126f4ef9a4713bc29e71874b5a32d"
format_stamp: "Formatted at 2026-08-09 16:24:25 on dist-test-slave-s11t"
server_key: "70b4a428b166f9f2826651a27629b4b2"
server_key_iv: "d1c26f337882deee384bd7a9eb8dfac8"
server_key_version: "encryptionkey@0"
I20260809 16:24:25.232535 20163 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786292417722232-21710-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786292417722232-21710-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786292417722232-21710-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:24:25.245306 20163 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:24:25.245575 20163 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:24:25.245724 20163 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:24:25.245939 20163 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:24:25.246264 20163 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:24:25.246320 20163 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:24:25.246361 20163 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:24:25.246388 20163 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:24:25.252233 20163 rpc_server.cc:307] RPC server started. Bound to: 127.21.51.131:35609
I20260809 16:24:25.252276 20348 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.51.131:35609 every 8 connection(s)
I20260809 16:24:25.252591 20163 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786292417722232-21710-0/minicluster-data/ts-2/data/info.pb
I20260809 16:24:25.258042 19781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:25.253378 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.51.131:39189 (local address 127.21.51.190:34021)
0809 16:24:25.253442 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0809 16:24:25.253445 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 16:24:25.253743 (+ 298us) server_negotiation.cc:407] Connection header received
0809 16:24:25.254323 (+ 580us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:24:25.254324 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:24:25.254337 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:24:25.254381 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:24:25.255166 (+ 785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:24:25.255671 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:24:25.256655 (+ 984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:24:25.256877 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:24:25.256970 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:24:25.257738 (+ 768us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:24:25.257740 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:24:25.257741 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 16:24:25.257745 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:24:25.257760 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 16:24:25.257765 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:24:25.257833 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0809 16:24:25.257928 (+ 95us) server_negotiation.cc:299] Negotiation successful
0809 16:24:25.257962 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260809 16:24:25.258251 20352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:25.253371 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:34021 (local address 127.21.51.131:39189)
0809 16:24:25.253573 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0809 16:24:25.253626 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0809 16:24:25.254221 (+ 595us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:24:25.254464 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:24:25.254474 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:24:25.254483 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:24:25.255068 (+ 585us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:24:25.255072 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:24:25.255769 (+ 697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:24:25.255770 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:24:25.256560 (+ 790us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:24:25.256562 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:24:25.256986 (+ 424us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:24:25.256987 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:24:25.257115 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:24:25.257601 (+ 486us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 16:24:25.257602 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:24:25.257629 (+ 27us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 16:24:25.257649 (+ 20us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:24:25.257835 (+ 186us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:24:25.257841 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:24:25.257844 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 16:24:25.257928 (+ 84us) client_negotiation.cc:240] Negotiation successful
0809 16:24:25.258025 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":90,"threads_started":1}
I20260809 16:24:25.258569 21710 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrNqUCn/build/release/bin/kudu as pid 20163
I20260809 16:24:25.258699 21710 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786292417722232-21710-0/minicluster-data/ts-2/wal/instance
I20260809 16:24:25.258883 20350 heartbeater.cc:347] Connected to a master server at 127.21.51.190:34021
I20260809 16:24:25.258982 20350 heartbeater.cc:464] Registering TS with master...
I20260809 16:24:25.258984 21710 external_mini_cluster.cc:1428] Setting key 5a9e8e029b4cd3d8a84c7b885c039e98
I20260809 16:24:25.259177 20350 heartbeater.cc:511] Master 127.21.51.190:34021 requested a full tablet report, sending...
I20260809 16:24:25.259631 19696 ts_manager.cc:194] Registered new tserver with Master: 81d126f4ef9a4713bc29e71874b5a32d (127.21.51.131:35609)
I20260809 16:24:25.260043 21710 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 16:24:25.260174 19696 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.21.51.131:39189
I20260809 16:24:25.264111 19781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:25.261446 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38048 (local address 127.21.51.190:34021)
0809 16:24:25.261484 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0809 16:24:25.261485 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 16:24:25.261641 (+ 156us) server_negotiation.cc:407] Connection header received
0809 16:24:25.261670 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:24:25.261671 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:24:25.261682 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:24:25.261743 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:24:25.261887 (+ 144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:24:25.262382 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:24:25.263122 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:24:25.263290 (+ 168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:24:25.263338 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:24:25.263961 (+ 623us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:24:25.263962 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:24:25.263962 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 16:24:25.263965 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:24:25.263974 (+ 9us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 16:24:25.263976 (+ 2us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:24:25.264012 (+ 36us) server_negotiation.cc:1035] Waiting for connection context
0809 16:24:25.264047 (+ 35us) server_negotiation.cc:299] Negotiation successful
0809 16:24:25.264064 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260809 16:24:25.265606 19696 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38048:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 16:24:25.267102 19696 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 16:24:25.273476 20359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:25.269297 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.130:38349 (local address 127.0.0.1:47990)
0809 16:24:25.269905 (+ 608us) negotiation.cc:107] Waiting for socket to connect
0809 16:24:25.269913 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0809 16:24:25.269955 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:24:25.270115 (+ 160us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:24:25.270116 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:24:25.270118 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:24:25.270221 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:24:25.270224 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:24:25.271183 (+ 959us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:24:25.271183 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:24:25.272279 (+ 1096us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:24:25.272282 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:24:25.273167 (+ 885us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:24:25.273168 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:24:25.273241 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:24:25.273254 (+ 13us) client_negotiation.cc:769] Sending connection context
0809 16:24:25.273338 (+ 84us) client_negotiation.cc:240] Negotiation successful
0809 16:24:25.273388 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":592,"mutex_wait_us":2,"thread_start_us":88,"threads_started":1}
I20260809 16:24:25.273480 20361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:25.269487 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47990 (local address 127.21.51.130:38349)
0809 16:24:25.269836 (+ 349us) server_negotiation.cc:206] Beginning negotiation
0809 16:24:25.269841 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 16:24:25.269967 (+ 126us) server_negotiation.cc:407] Connection header received
0809 16:24:25.270018 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:24:25.270020 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:24:25.270036 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:24:25.270118 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:24:25.270311 (+ 193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:24:25.271105 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:24:25.272371 (+ 1266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:24:25.273074 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:24:25.273152 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:24:25.273241 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0809 16:24:25.273354 (+ 113us) server_negotiation.cc:299] Negotiation successful
0809 16:24:25.273397 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":305,"thread_start_us":77,"threads_started":1}
I20260809 16:24:25.273873 20358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:25.269297 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.131:35609 (local address 127.0.0.1:45030)
0809 16:24:25.269661 (+ 364us) negotiation.cc:107] Waiting for socket to connect
0809 16:24:25.269702 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0809 16:24:25.269785 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:24:25.270080 (+ 295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:24:25.270083 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:24:25.270085 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:24:25.270145 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:24:25.270147 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:24:25.271418 (+ 1271us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:24:25.271419 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:24:25.272553 (+ 1134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:24:25.272555 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:24:25.273537 (+ 982us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:24:25.273539 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:24:25.273655 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:24:25.273658 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 16:24:25.273745 (+ 87us) client_negotiation.cc:240] Negotiation successful
0809 16:24:25.273805 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":336,"mutex_wait_us":2,"thread_start_us":58,"threads_started":1}
I20260809 16:24:25.273892 20360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:25.269376 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45030 (local address 127.21.51.131:35609)
0809 16:24:25.269640 (+ 264us) server_negotiation.cc:206] Beginning negotiation
0809 16:24:25.269644 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 16:24:25.269786 (+ 142us) server_negotiation.cc:407] Connection header received
0809 16:24:25.269875 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:24:25.269876 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:24:25.269907 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:24:25.270006 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:24:25.270236 (+ 230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:24:25.271317 (+ 1081us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:24:25.272640 (+ 1323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:24:25.273344 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:24:25.273419 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:24:25.273497 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0809 16:24:25.273764 (+ 267us) server_negotiation.cc:299] Negotiation successful
0809 16:24:25.273804 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":227,"thread_start_us":100,"threads_started":1}
I20260809 16:24:25.273957 20363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:25.269601 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53278 (local address 127.21.51.129:33047)
0809 16:24:25.269777 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0809 16:24:25.269782 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 16:24:25.270260 (+ 478us) server_negotiation.cc:407] Connection header received
0809 16:24:25.270332 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:24:25.270334 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:24:25.270349 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:24:25.270435 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:24:25.270546 (+ 111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:24:25.271593 (+ 1047us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:24:25.272815 (+ 1222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:24:25.273449 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:24:25.273515 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:24:25.273613 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0809 16:24:25.273803 (+ 190us) server_negotiation.cc:299] Negotiation successful
0809 16:24:25.273849 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":125,"thread_start_us":94,"threads_started":1}
I20260809 16:24:25.274509 20248 tablet_service.cc:1467] Processing CreateTablet for tablet 8919ec5ae5214d9883844f41abdea59f (DEFAULT_TABLE table=test-table [id=9158594b1ce74109bc663f0c804b2ca8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:24:25.274509 20072 tablet_service.cc:1467] Processing CreateTablet for tablet 8919ec5ae5214d9883844f41abdea59f (DEFAULT_TABLE table=test-table [id=9158594b1ce74109bc663f0c804b2ca8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:24:25.274881 20248 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8919ec5ae5214d9883844f41abdea59f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:24:25.274891 20072 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8919ec5ae5214d9883844f41abdea59f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:24:25.278257 20364 tablet_bootstrap.cc:492] T 8919ec5ae5214d9883844f41abdea59f P 81d126f4ef9a4713bc29e71874b5a32d: Bootstrap starting.
I20260809 16:24:25.278340 20362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:25.269497 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.129:33047 (local address 127.0.0.1:53278)
0809 16:24:25.270200 (+ 703us) negotiation.cc:107] Waiting for socket to connect
0809 16:24:25.270210 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 16:24:25.270262 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:24:25.270454 (+ 192us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:24:25.270454 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:24:25.270456 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:24:25.270489 (+ 33us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:24:25.270490 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:24:25.271676 (+ 1186us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:24:25.271677 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:24:25.272754 (+ 1077us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:24:25.272756 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:24:25.273652 (+ 896us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:24:25.273653 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:24:25.273710 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:24:25.273714 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 16:24:25.273760 (+ 46us) client_negotiation.cc:240] Negotiation successful
0809 16:24:25.278270 (+ 4510us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":684,"mutex_wait_us":4455,"thread_start_us":77,"threads_started":1}
I20260809 16:24:25.279098 20364 tablet_bootstrap.cc:654] T 8919ec5ae5214d9883844f41abdea59f P 81d126f4ef9a4713bc29e71874b5a32d: Neither blocks nor log segments found. Creating new log.
I20260809 16:24:25.279095 19873 tablet_service.cc:1467] Processing CreateTablet for tablet 8919ec5ae5214d9883844f41abdea59f (DEFAULT_TABLE table=test-table [id=9158594b1ce74109bc663f0c804b2ca8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:24:25.279430 20364 log.cc:824] T 8919ec5ae5214d9883844f41abdea59f P 81d126f4ef9a4713bc29e71874b5a32d: Log is configured to *not* fsync() on all Append() calls
I20260809 16:24:25.279464 19873 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8919ec5ae5214d9883844f41abdea59f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:24:25.279848 20365 tablet_bootstrap.cc:492] T 8919ec5ae5214d9883844f41abdea59f P 242e85ecda274ead8ba6ad25046e8bc8: Bootstrap starting.
I20260809 16:24:25.280313 20364 tablet_bootstrap.cc:492] T 8919ec5ae5214d9883844f41abdea59f P 81d126f4ef9a4713bc29e71874b5a32d: No bootstrap required, opened a new log
I20260809 16:24:25.280386 20364 ts_tablet_manager.cc:1397] T 8919ec5ae5214d9883844f41abdea59f P 81d126f4ef9a4713bc29e71874b5a32d: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 16:24:25.281186 20365 tablet_bootstrap.cc:654] T 8919ec5ae5214d9883844f41abdea59f P 242e85ecda274ead8ba6ad25046e8bc8: Neither blocks nor log segments found. Creating new log.
I20260809 16:24:25.281493 20365 log.cc:824] T 8919ec5ae5214d9883844f41abdea59f P 242e85ecda274ead8ba6ad25046e8bc8: Log is configured to *not* fsync() on all Append() calls
I20260809 16:24:25.282219 20365 tablet_bootstrap.cc:492] T 8919ec5ae5214d9883844f41abdea59f P 242e85ecda274ead8ba6ad25046e8bc8: No bootstrap required, opened a new log
I20260809 16:24:25.282294 20365 ts_tablet_manager.cc:1397] T 8919ec5ae5214d9883844f41abdea59f P 242e85ecda274ead8ba6ad25046e8bc8: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 16:24:25.283042 20364 raft_consensus.cc:348] T 8919ec5ae5214d9883844f41abdea59f P 81d126f4ef9a4713bc29e71874b5a32d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "242e85ecda274ead8ba6ad25046e8bc8" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 38349 } } peers { permanent_uuid: "81d126f4ef9a4713bc29e71874b5a32d" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 35609 } } peers { permanent_uuid: "3f31b397033a4e7baea6521846e46690" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 33047 } }
I20260809 16:24:25.283223 20364 raft_consensus.cc:374] T 8919ec5ae5214d9883844f41abdea59f P 81d126f4ef9a4713bc29e71874b5a32d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:24:25.283272 20364 raft_consensus.cc:729] T 8919ec5ae5214d9883844f41abdea59f P 81d126f4ef9a4713bc29e71874b5a32d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 81d126f4ef9a4713bc29e71874b5a32d, State: Initialized, Role: FOLLOWER
I20260809 16:24:25.283393 20364 consensus_queue.cc:260] T 8919ec5ae5214d9883844f41abdea59f P 81d126f4ef9a4713bc29e71874b5a32d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "242e85ecda274ead8ba6ad25046e8bc8" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 38349 } } peers { permanent_uuid: "81d126f4ef9a4713bc29e71874b5a32d" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 35609 } } peers { permanent_uuid: "3f31b397033a4e7baea6521846e46690" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 33047 } }
I20260809 16:24:25.283659 20364 ts_tablet_manager.cc:1428] T 8919ec5ae5214d9883844f41abdea59f P 81d126f4ef9a4713bc29e71874b5a32d: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260809 16:24:25.283694 20350 heartbeater.cc:503] Master 127.21.51.190:34021 was elected leader, sending a full tablet report...
I20260809 16:24:25.283850 20370 tablet_bootstrap.cc:492] T 8919ec5ae5214d9883844f41abdea59f P 3f31b397033a4e7baea6521846e46690: Bootstrap starting.
I20260809 16:24:25.284502 20370 tablet_bootstrap.cc:654] T 8919ec5ae5214d9883844f41abdea59f P 3f31b397033a4e7baea6521846e46690: Neither blocks nor log segments found. Creating new log.
I20260809 16:24:25.284857 20370 log.cc:824] T 8919ec5ae5214d9883844f41abdea59f P 3f31b397033a4e7baea6521846e46690: Log is configured to *not* fsync() on all Append() calls
I20260809 16:24:25.284780 20365 raft_consensus.cc:348] T 8919ec5ae5214d9883844f41abdea59f P 242e85ecda274ead8ba6ad25046e8bc8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "242e85ecda274ead8ba6ad25046e8bc8" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 38349 } } peers { permanent_uuid: "81d126f4ef9a4713bc29e71874b5a32d" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 35609 } } peers { permanent_uuid: "3f31b397033a4e7baea6521846e46690" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 33047 } }
I20260809 16:24:25.284960 20365 raft_consensus.cc:374] T 8919ec5ae5214d9883844f41abdea59f P 242e85ecda274ead8ba6ad25046e8bc8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:24:25.285010 20365 raft_consensus.cc:729] T 8919ec5ae5214d9883844f41abdea59f P 242e85ecda274ead8ba6ad25046e8bc8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 242e85ecda274ead8ba6ad25046e8bc8, State: Initialized, Role: FOLLOWER
I20260809 16:24:25.285125 20365 consensus_queue.cc:260] T 8919ec5ae5214d9883844f41abdea59f P 242e85ecda274ead8ba6ad25046e8bc8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "242e85ecda274ead8ba6ad25046e8bc8" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 38349 } } peers { permanent_uuid: "81d126f4ef9a4713bc29e71874b5a32d" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 35609 } } peers { permanent_uuid: "3f31b397033a4e7baea6521846e46690" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 33047 } }
I20260809 16:24:25.285396 20365 ts_tablet_manager.cc:1428] T 8919ec5ae5214d9883844f41abdea59f P 242e85ecda274ead8ba6ad25046e8bc8: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 16:24:25.285456 20160 heartbeater.cc:503] Master 127.21.51.190:34021 was elected leader, sending a full tablet report...
I20260809 16:24:25.285647 20370 tablet_bootstrap.cc:492] T 8919ec5ae5214d9883844f41abdea59f P 3f31b397033a4e7baea6521846e46690: No bootstrap required, opened a new log
I20260809 16:24:25.285723 20370 ts_tablet_manager.cc:1397] T 8919ec5ae5214d9883844f41abdea59f P 3f31b397033a4e7baea6521846e46690: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260809 16:24:25.287799 20370 raft_consensus.cc:348] T 8919ec5ae5214d9883844f41abdea59f P 3f31b397033a4e7baea6521846e46690 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "242e85ecda274ead8ba6ad25046e8bc8" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 38349 } } peers { permanent_uuid: "81d126f4ef9a4713bc29e71874b5a32d" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 35609 } } peers { permanent_uuid: "3f31b397033a4e7baea6521846e46690" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 33047 } }
I20260809 16:24:25.287921 20370 raft_consensus.cc:374] T 8919ec5ae5214d9883844f41abdea59f P 3f31b397033a4e7baea6521846e46690 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:24:25.287943 20370 raft_consensus.cc:729] T 8919ec5ae5214d9883844f41abdea59f P 3f31b397033a4e7baea6521846e46690 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f31b397033a4e7baea6521846e46690, State: Initialized, Role: FOLLOWER
I20260809 16:24:25.288010 20370 consensus_queue.cc:260] T 8919ec5ae5214d9883844f41abdea59f P 3f31b397033a4e7baea6521846e46690 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "242e85ecda274ead8ba6ad25046e8bc8" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 38349 } } peers { permanent_uuid: "81d126f4ef9a4713bc29e71874b5a32d" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 35609 } } peers { permanent_uuid: "3f31b397033a4e7baea6521846e46690" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 33047 } }
I20260809 16:24:25.288190 20370 ts_tablet_manager.cc:1428] T 8919ec5ae5214d9883844f41abdea59f P 3f31b397033a4e7baea6521846e46690: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260809 16:24:25.288307 19967 heartbeater.cc:503] Master 127.21.51.190:34021 was elected leader, sending a full tablet report...
I20260809 16:24:25.320955 20373 raft_consensus.cc:482] T 8919ec5ae5214d9883844f41abdea59f P 242e85ecda274ead8ba6ad25046e8bc8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 16:24:25.321066 20373 raft_consensus.cc:504] T 8919ec5ae5214d9883844f41abdea59f P 242e85ecda274ead8ba6ad25046e8bc8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "242e85ecda274ead8ba6ad25046e8bc8" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 38349 } } peers { permanent_uuid: "81d126f4ef9a4713bc29e71874b5a32d" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 35609 } } peers { permanent_uuid: "3f31b397033a4e7baea6521846e46690" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 33047 } }
I20260809 16:24:25.321377 20373 leader_election.cc:288] T 8919ec5ae5214d9883844f41abdea59f P 242e85ecda274ead8ba6ad25046e8bc8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 81d126f4ef9a4713bc29e71874b5a32d (127.21.51.131:35609), 3f31b397033a4e7baea6521846e46690 (127.21.51.129:33047)
I20260809 16:24:25.324369 20162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:25.321562 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.131:35609 (local address 127.21.51.130:53711)
0809 16:24:25.321639 (+ 77us) negotiation.cc:107] Waiting for socket to connect
0809 16:24:25.321658 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 16:24:25.321717 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:24:25.321874 (+ 157us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:24:25.321875 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:24:25.321877 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:24:25.321979 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:24:25.321981 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:24:25.322714 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:24:25.322716 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:24:25.323545 (+ 829us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:24:25.323547 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:24:25.324101 (+ 554us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:24:25.324103 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:24:25.324178 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:24:25.324181 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 16:24:25.324239 (+ 58us) client_negotiation.cc:240] Negotiation successful
0809 16:24:25.324273 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":50,"mutex_wait_us":38}
I20260809 16:24:25.324375 20360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:25.321653 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.51.130:53711 (local address 127.21.51.131:35609)
0809 16:24:25.321704 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0809 16:24:25.321706 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:24:25.321717 (+ 11us) server_negotiation.cc:407] Connection header received
0809 16:24:25.321780 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:24:25.321781 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:24:25.321790 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:24:25.321857 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:24:25.322078 (+ 221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:24:25.322624 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:24:25.323643 (+ 1019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:24:25.324019 (+ 376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:24:25.324075 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:24:25.324081 (+ 6us) server_negotiation.cc:1035] Waiting for connection context
0809 16:24:25.324254 (+ 173us) server_negotiation.cc:299] Negotiation successful
0809 16:24:25.324304 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260809 16:24:25.324460 20363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:25.321782 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.51.130:44427 (local address 127.21.51.129:33047)
0809 16:24:25.321827 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0809 16:24:25.321830 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 16:24:25.321957 (+ 127us) server_negotiation.cc:407] Connection header received
0809 16:24:25.321987 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:24:25.321988 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:24:25.322001 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:24:25.322057 (+ 56us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:24:25.322214 (+ 157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:24:25.322682 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:24:25.323680 (+ 998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:24:25.324040 (+ 360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:24:25.324136 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:24:25.324144 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0809 16:24:25.324344 (+ 200us) server_negotiation.cc:299] Negotiation successful
0809 16:24:25.324386 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260809 16:24:25.324697 20376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:25.321692 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.129:33047 (local address 127.21.51.130:44427)
0809 16:24:25.321880 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0809 16:24:25.321889 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 16:24:25.321945 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:24:25.322077 (+ 132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:24:25.322078 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:24:25.322079 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:24:25.322147 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:24:25.322149 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:24:25.322799 (+ 650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:24:25.322800 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:24:25.323558 (+ 758us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:24:25.323560 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:24:25.324163 (+ 603us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:24:25.324164 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:24:25.324258 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:24:25.324262 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 16:24:25.324318 (+ 56us) client_negotiation.cc:240] Negotiation successful
0809 16:24:25.324353 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":118,"threads_started":1}
I20260809 16:24:25.324781 20276 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8919ec5ae5214d9883844f41abdea59f" candidate_uuid: "242e85ecda274ead8ba6ad25046e8bc8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "81d126f4ef9a4713bc29e71874b5a32d" is_pre_election: true
I20260809 16:24:25.324981 20276 raft_consensus.cc:2454] T 8919ec5ae5214d9883844f41abdea59f P 81d126f4ef9a4713bc29e71874b5a32d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 242e85ecda274ead8ba6ad25046e8bc8 in term 0.
I20260809 16:24:25.325070 19902 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8919ec5ae5214d9883844f41abdea59f" candidate_uuid: "242e85ecda274ead8ba6ad25046e8bc8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f31b397033a4e7baea6521846e46690" is_pre_election: true
I20260809 16:24:25.325222 19902 raft_consensus.cc:2454] T 8919ec5ae5214d9883844f41abdea59f P 3f31b397033a4e7baea6521846e46690 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 242e85ecda274ead8ba6ad25046e8bc8 in term 0.
I20260809 16:24:25.325251 19996 leader_election.cc:302] T 8919ec5ae5214d9883844f41abdea59f P 242e85ecda274ead8ba6ad25046e8bc8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 242e85ecda274ead8ba6ad25046e8bc8, 81d126f4ef9a4713bc29e71874b5a32d; no voters:
I20260809 16:24:25.325394 20373 raft_consensus.cc:2781] T 8919ec5ae5214d9883844f41abdea59f P 242e85ecda274ead8ba6ad25046e8bc8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 16:24:25.325465 20373 raft_consensus.cc:482] T 8919ec5ae5214d9883844f41abdea59f P 242e85ecda274ead8ba6ad25046e8bc8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 16:24:25.325502 20373 raft_consensus.cc:3037] T 8919ec5ae5214d9883844f41abdea59f P 242e85ecda274ead8ba6ad25046e8bc8 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:24:25.326211 20373 raft_consensus.cc:504] T 8919ec5ae5214d9883844f41abdea59f P 242e85ecda274ead8ba6ad25046e8bc8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "242e85ecda274ead8ba6ad25046e8bc8" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 38349 } } peers { permanent_uuid: "81d126f4ef9a4713bc29e71874b5a32d" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 35609 } } peers { permanent_uuid: "3f31b397033a4e7baea6521846e46690" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 33047 } }
I20260809 16:24:25.326351 20373 leader_election.cc:288] T 8919ec5ae5214d9883844f41abdea59f P 242e85ecda274ead8ba6ad25046e8bc8 [CANDIDATE]: Term 1 election: Requested vote from peers 81d126f4ef9a4713bc29e71874b5a32d (127.21.51.131:35609), 3f31b397033a4e7baea6521846e46690 (127.21.51.129:33047)
I20260809 16:24:25.326531 20276 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8919ec5ae5214d9883844f41abdea59f" candidate_uuid: "242e85ecda274ead8ba6ad25046e8bc8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "81d126f4ef9a4713bc29e71874b5a32d"
I20260809 16:24:25.326541 19902 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8919ec5ae5214d9883844f41abdea59f" candidate_uuid: "242e85ecda274ead8ba6ad25046e8bc8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f31b397033a4e7baea6521846e46690"
I20260809 16:24:25.326622 20276 raft_consensus.cc:3037] T 8919ec5ae5214d9883844f41abdea59f P 81d126f4ef9a4713bc29e71874b5a32d [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:24:25.326610 19902 raft_consensus.cc:3037] T 8919ec5ae5214d9883844f41abdea59f P 3f31b397033a4e7baea6521846e46690 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:24:25.327303 20276 raft_consensus.cc:2454] T 8919ec5ae5214d9883844f41abdea59f P 81d126f4ef9a4713bc29e71874b5a32d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 242e85ecda274ead8ba6ad25046e8bc8 in term 1.
I20260809 16:24:25.327410 19902 raft_consensus.cc:2454] T 8919ec5ae5214d9883844f41abdea59f P 3f31b397033a4e7baea6521846e46690 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 242e85ecda274ead8ba6ad25046e8bc8 in term 1.
I20260809 16:24:25.327517 19996 leader_election.cc:302] T 8919ec5ae5214d9883844f41abdea59f P 242e85ecda274ead8ba6ad25046e8bc8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 242e85ecda274ead8ba6ad25046e8bc8, 81d126f4ef9a4713bc29e71874b5a32d; no voters:
I20260809 16:24:25.327623 20373 raft_consensus.cc:2781] T 8919ec5ae5214d9883844f41abdea59f P 242e85ecda274ead8ba6ad25046e8bc8 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:24:25.327798 20373 raft_consensus.cc:686] T 8919ec5ae5214d9883844f41abdea59f P 242e85ecda274ead8ba6ad25046e8bc8 [term 1 LEADER]: Becoming Leader. State: Replica: 242e85ecda274ead8ba6ad25046e8bc8, State: Running, Role: LEADER
I20260809 16:24:25.327911 20373 consensus_queue.cc:237] T 8919ec5ae5214d9883844f41abdea59f P 242e85ecda274ead8ba6ad25046e8bc8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "242e85ecda274ead8ba6ad25046e8bc8" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 38349 } } peers { permanent_uuid: "81d126f4ef9a4713bc29e71874b5a32d" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 35609 } } peers { permanent_uuid: "3f31b397033a4e7baea6521846e46690" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 33047 } }
I20260809 16:24:25.328814 19696 catalog_manager.cc:5495] T 8919ec5ae5214d9883844f41abdea59f P 242e85ecda274ead8ba6ad25046e8bc8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 242e85ecda274ead8ba6ad25046e8bc8 (127.21.51.130). New cstate: current_term: 1 leader_uuid: "242e85ecda274ead8ba6ad25046e8bc8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "242e85ecda274ead8ba6ad25046e8bc8" member_type: VOTER last_known_addr { host: "127.21.51.130" port: 38349 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "81d126f4ef9a4713bc29e71874b5a32d" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 35609 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3f31b397033a4e7baea6521846e46690" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 33047 } health_report { overall_health: UNKNOWN } } }
I20260809 16:24:25.348273 20361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:25.345856 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48000 (local address 127.21.51.130:38349)
0809 16:24:25.345910 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0809 16:24:25.345912 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:24:25.345940 (+ 28us) server_negotiation.cc:407] Connection header received
0809 16:24:25.346032 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:24:25.346033 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:24:25.346051 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:24:25.346139 (+ 88us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 16:24:25.346270 (+ 131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:24:25.346734 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:24:25.347478 (+ 744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:24:25.347686 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:24:25.347759 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:24:25.347934 (+ 175us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 16:24:25.348037 (+ 103us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 16:24:25.348122 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0809 16:24:25.348163 (+ 41us) server_negotiation.cc:299] Negotiation successful
0809 16:24:25.348180 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260809 16:24:25.350001 20276 raft_consensus.cc:1261] T 8919ec5ae5214d9883844f41abdea59f P 81d126f4ef9a4713bc29e71874b5a32d [term 1 FOLLOWER]: Refusing update from remote peer 242e85ecda274ead8ba6ad25046e8bc8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 16:24:25.349995 19902 raft_consensus.cc:1261] T 8919ec5ae5214d9883844f41abdea59f P 3f31b397033a4e7baea6521846e46690 [term 1 FOLLOWER]: Refusing update from remote peer 242e85ecda274ead8ba6ad25046e8bc8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 16:24:25.350320 20373 consensus_queue.cc:1035] T 8919ec5ae5214d9883844f41abdea59f P 242e85ecda274ead8ba6ad25046e8bc8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "81d126f4ef9a4713bc29e71874b5a32d" member_type: VOTER last_known_addr { host: "127.21.51.131" port: 35609 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 16:24:25.350456 20377 consensus_queue.cc:1035] T 8919ec5ae5214d9883844f41abdea59f P 242e85ecda274ead8ba6ad25046e8bc8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f31b397033a4e7baea6521846e46690" member_type: VOTER last_known_addr { host: "127.21.51.129" port: 33047 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 16:24:25.352910 20382 mvcc.cc:205] Tried to move back new op lower bound from 7316654757271150592 to 7316654757183864832. Current Snapshot: MvccSnapshot[applied={T|T < 7316654757271150592}]
I20260809 16:24:25.353768 20386 mvcc.cc:205] Tried to move back new op lower bound from 7316654757271150592 to 7316654757183864832. Current Snapshot: MvccSnapshot[applied={T|T < 7316654757271150592 or (T in {7316654757271150592})}]
I20260809 16:24:25.355361 20385 mvcc.cc:205] Tried to move back new op lower bound from 7316654757271150592 to 7316654757183864832. Current Snapshot: MvccSnapshot[applied={T|T < 7316654757271150592 or (T in {7316654757271150592})}]
I20260809 16:24:25.355404 19696 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38048:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 16:24:25.355515 19696 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38048:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 16:24:25.356231 19696 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 603e269124fe45c88d9278d0cdc2e5f1: Sending DeleteTablet for 3 replicas of tablet 8919ec5ae5214d9883844f41abdea59f
I20260809 16:24:25.356567 20248 tablet_service.cc:1514] Processing DeleteTablet for tablet 8919ec5ae5214d9883844f41abdea59f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:24:25 UTC) from {username='slave'} at 127.0.0.1:45030
I20260809 16:24:25.356607 19873 tablet_service.cc:1514] Processing DeleteTablet for tablet 8919ec5ae5214d9883844f41abdea59f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:24:25 UTC) from {username='slave'} at 127.0.0.1:53278
I20260809 16:24:25.356882 20072 tablet_service.cc:1514] Processing DeleteTablet for tablet 8919ec5ae5214d9883844f41abdea59f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:24:25 UTC) from {username='slave'} at 127.0.0.1:47990
I20260809 16:24:25.356945 20393 tablet_replica.cc:330] T 8919ec5ae5214d9883844f41abdea59f P 81d126f4ef9a4713bc29e71874b5a32d: stopping tablet replica
I20260809 16:24:25.357023 21710 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrNqUCn/build/release/bin/kudu with pid 19782
I20260809 16:24:25.357040 20393 raft_consensus.cc:2229] T 8919ec5ae5214d9883844f41abdea59f P 81d126f4ef9a4713bc29e71874b5a32d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 16:24:25.357131 20393 raft_consensus.cc:2258] T 8919ec5ae5214d9883844f41abdea59f P 81d126f4ef9a4713bc29e71874b5a32d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 16:24:25.368259 20396 tablet_replica.cc:330] T 8919ec5ae5214d9883844f41abdea59f P 242e85ecda274ead8ba6ad25046e8bc8: stopping tablet replica
I20260809 16:24:25.368462 20396 raft_consensus.cc:2229] T 8919ec5ae5214d9883844f41abdea59f P 242e85ecda274ead8ba6ad25046e8bc8 [term 1 LEADER]: Raft consensus shutting down.
I20260809 16:24:25.368589 20393 ts_tablet_manager.cc:1905] T 8919ec5ae5214d9883844f41abdea59f P 81d126f4ef9a4713bc29e71874b5a32d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 16:24:25.368623 20396 raft_consensus.cc:2258] T 8919ec5ae5214d9883844f41abdea59f P 242e85ecda274ead8ba6ad25046e8bc8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 16:24:25.369225 20396 ts_tablet_manager.cc:1905] T 8919ec5ae5214d9883844f41abdea59f P 242e85ecda274ead8ba6ad25046e8bc8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 16:24:25.370685 20393 ts_tablet_manager.cc:1918] T 8919ec5ae5214d9883844f41abdea59f P 81d126f4ef9a4713bc29e71874b5a32d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 16:24:25.370770 20393 log.cc:1196] T 8919ec5ae5214d9883844f41abdea59f P 81d126f4ef9a4713bc29e71874b5a32d: Deleting WAL directory at /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786292417722232-21710-0/minicluster-data/ts-2/wal/wals/8919ec5ae5214d9883844f41abdea59f
I20260809 16:24:25.371048 20393 ts_tablet_manager.cc:1939] T 8919ec5ae5214d9883844f41abdea59f P 81d126f4ef9a4713bc29e71874b5a32d: Deleting consensus metadata
I20260809 16:24:25.371237 20396 ts_tablet_manager.cc:1918] T 8919ec5ae5214d9883844f41abdea59f P 242e85ecda274ead8ba6ad25046e8bc8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 16:24:25.371357 20396 log.cc:1196] T 8919ec5ae5214d9883844f41abdea59f P 242e85ecda274ead8ba6ad25046e8bc8: Deleting WAL directory at /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786292417722232-21710-0/minicluster-data/ts-1/wal/wals/8919ec5ae5214d9883844f41abdea59f
I20260809 16:24:25.371659 19669 catalog_manager.cc:4841] TS 81d126f4ef9a4713bc29e71874b5a32d (127.21.51.131:35609): tablet 8919ec5ae5214d9883844f41abdea59f (table test-table [id=9158594b1ce74109bc663f0c804b2ca8]) successfully deleted
I20260809 16:24:25.371697 20396 ts_tablet_manager.cc:1939] T 8919ec5ae5214d9883844f41abdea59f P 242e85ecda274ead8ba6ad25046e8bc8: Deleting consensus metadata
I20260809 16:24:25.372164 19673 catalog_manager.cc:4841] TS 242e85ecda274ead8ba6ad25046e8bc8 (127.21.51.130:38349): tablet 8919ec5ae5214d9883844f41abdea59f (table test-table [id=9158594b1ce74109bc663f0c804b2ca8]) successfully deleted
W20260809 16:24:25.373656 19669 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.21.51.129:33047 (error 108)
I20260809 16:24:25.373899 21710 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrNqUCn/build/release/bin/kudu with pid 19972
I20260809 16:24:25.379856 20358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:25.374042 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.129:33047 (local address 127.0.0.1:53290)
0809 16:24:25.379741 (+ 5699us) negotiation.cc:107] Waiting for socket to connect
0809 16:24:25.379814 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.51.129:33047: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":5663}
W20260809 16:24:25.380033 19669 catalog_manager.cc:4584] TS 3f31b397033a4e7baea6521846e46690 (127.21.51.129:33047): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 8919ec5ae5214d9883844f41abdea59f: Network error: Client connection negotiation failed: client connection to 127.21.51.129:33047: connect: Connection refused (error 111)
I20260809 16:24:25.385581 21710 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrNqUCn/build/release/bin/kudu with pid 20163
I20260809 16:24:25.399923 21710 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrNqUCn/build/release/bin/kudu with pid 19645
2026-08-09T16:24:25Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (710 ms)
[----------] 6 tests from EncryptionPolicyTest (9281 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786292417722232-21710-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 16:24:25 dist-test-slave-s11t krb5kdc[20403](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 16:24:25 dist-test-slave-s11t krb5kdc[20403](info): set up 2 sockets
Aug 09 16:24:25 dist-test-slave-s11t krb5kdc[20403](info): commencing operation
krb5kdc: starting...
W20260809 16:24:26.650727 21710 mini_kdc.cc:121] Time spent starting KDC: real 1.227s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786292417722232-21710-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786292417722232-21710-0/krb5kdc/test-user.keytab.
Aug 09 16:24:26 dist-test-slave-s11t krb5kdc[20403](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292666, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T16:24:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T16:24:26Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786292417722232-21710-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786292417722232-21710-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 16:24:26.775640 21710 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrNqUCn/build/release/bin/kudu
/tmp/dist-test-taskrNqUCn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786292417722232-21710-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786292417722232-21710-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786292417722232-21710-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786292417722232-21710-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44935
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786292417722232-21710-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40143
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44935
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786292417722232-21710-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786292417722232-21710-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786292417722232-21710-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:24:26.858742 20419 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:24:26.858985 20419 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:24:26.859025 20419 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:24:26.860168 20419 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 16:24:26.860241 20419 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:24:26.860265 20419 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:24:26.860287 20419 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 16:24:26.860316 20419 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 16:24:26.861860 20419 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40143
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786292417722232-21710-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786292417722232-21710-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44935
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786292417722232-21710-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44935
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786292417722232-21710-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20419
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786292417722232-21710-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:24:26.862182 20419 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:24:26.862437 20419 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:24:26.867579 20429 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:24:26.867714 20419 server_base.cc:1034] running on GCE node
W20260809 16:24:26.867506 20433 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:24:26.867504 20435 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:24:26.868147 20419 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:24:26.868432 20419 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:24:26.869572 20419 hybrid_clock.cc:648] HybridClock initialized: now 1786292666869561 us; error 33 us; skew 500 ppm
Aug 09 16:24:26 dist-test-slave-s11t krb5kdc[20403](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292666, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:24:26.876533 20419 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 16:24:26.877104 20419 webserver.cc:460] Webserver started at http://127.0.0.1:34635/ using document root <none> and password file <none>
I20260809 16:24:26.877353 20419 fs_manager.cc:359] Metadata directory not provided
I20260809 16:24:26.877424 20419 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:24:26.877578 20419 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:24:26.878654 20419 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786292417722232-21710-0/minicluster-data/master-0/data/instance:
uuid: "ab5a0a0f32c249e1a04422a4b14b680e"
format_stamp: "Formatted at 2026-08-09 16:24:26 on dist-test-slave-s11t"
server_key: "a1582a522fea45b3caae583a15485ba9"
server_key_iv: "ce6e1ae9136df1ee06e40f2e40a9d262"
server_key_version: "encryptionkey@0"
I20260809 16:24:26.879024 20419 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786292417722232-21710-0/minicluster-data/master-0/wal/instance:
uuid: "ab5a0a0f32c249e1a04422a4b14b680e"
format_stamp: "Formatted at 2026-08-09 16:24:26 on dist-test-slave-s11t"
server_key: "a1582a522fea45b3caae583a15485ba9"
server_key_iv: "ce6e1ae9136df1ee06e40f2e40a9d262"
server_key_version: "encryptionkey@0"
I20260809 16:24:26.880385 20419 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 16:24:26.881305 20445 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:24:26.881543 20419 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260809 16:24:26.881652 20419 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786292417722232-21710-0/minicluster-data/master-0/data,/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786292417722232-21710-0/minicluster-data/master-0/wal
uuid: "ab5a0a0f32c249e1a04422a4b14b680e"
format_stamp: "Formatted at 2026-08-09 16:24:26 on dist-test-slave-s11t"
server_key: "a1582a522fea45b3caae583a15485ba9"
server_key_iv: "ce6e1ae9136df1ee06e40f2e40a9d262"
server_key_version: "encryptionkey@0"
I20260809 16:24:26.881732 20419 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786292417722232-21710-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786292417722232-21710-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786292417722232-21710-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:24:26.897851 20419 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:24:26.899770 20419 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:24:26.899941 20419 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:24:26.904238 20419 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44935
I20260809 16:24:26.904273 20522 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44935 every 8 connection(s)
I20260809 16:24:26.904635 20419 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786292417722232-21710-0/minicluster-data/master-0/data/info.pb
I20260809 16:24:26.905539 20524 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:24:26.907771 20524 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ab5a0a0f32c249e1a04422a4b14b680e: Bootstrap starting.
I20260809 16:24:26.908310 20524 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ab5a0a0f32c249e1a04422a4b14b680e: Neither blocks nor log segments found. Creating new log.
I20260809 16:24:26.908571 20524 log.cc:824] T 00000000000000000000000000000000 P ab5a0a0f32c249e1a04422a4b14b680e: Log is configured to *not* fsync() on all Append() calls
I20260809 16:24:26.909224 20524 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ab5a0a0f32c249e1a04422a4b14b680e: No bootstrap required, opened a new log
I20260809 16:24:26.910741 20524 raft_consensus.cc:348] T 00000000000000000000000000000000 P ab5a0a0f32c249e1a04422a4b14b680e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab5a0a0f32c249e1a04422a4b14b680e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44935 } }
I20260809 16:24:26.910898 20524 raft_consensus.cc:374] T 00000000000000000000000000000000 P ab5a0a0f32c249e1a04422a4b14b680e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:24:26.910923 20524 raft_consensus.cc:729] T 00000000000000000000000000000000 P ab5a0a0f32c249e1a04422a4b14b680e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ab5a0a0f32c249e1a04422a4b14b680e, State: Initialized, Role: FOLLOWER
I20260809 16:24:26.911067 20524 consensus_queue.cc:260] T 00000000000000000000000000000000 P ab5a0a0f32c249e1a04422a4b14b680e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab5a0a0f32c249e1a04422a4b14b680e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44935 } }
I20260809 16:24:26.911149 20524 raft_consensus.cc:388] T 00000000000000000000000000000000 P ab5a0a0f32c249e1a04422a4b14b680e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 16:24:26.911154 21710 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrNqUCn/build/release/bin/kudu as pid 20419
I20260809 16:24:26.911192 20524 raft_consensus.cc:482] T 00000000000000000000000000000000 P ab5a0a0f32c249e1a04422a4b14b680e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 16:24:26.911234 20524 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ab5a0a0f32c249e1a04422a4b14b680e [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:24:26.911291 21710 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786292417722232-21710-0/minicluster-data/master-0/wal/instance
I20260809 16:24:26.911522 21710 external_mini_cluster.cc:1428] Setting key 8b72007805c06f99e08472103f627183
I20260809 16:24:26.911902 20524 raft_consensus.cc:504] T 00000000000000000000000000000000 P ab5a0a0f32c249e1a04422a4b14b680e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab5a0a0f32c249e1a04422a4b14b680e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44935 } }
I20260809 16:24:26.912093 20524 leader_election.cc:302] T 00000000000000000000000000000000 P ab5a0a0f32c249e1a04422a4b14b680e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ab5a0a0f32c249e1a04422a4b14b680e; no voters:
I20260809 16:24:26.912276 20524 leader_election.cc:288] T 00000000000000000000000000000000 P ab5a0a0f32c249e1a04422a4b14b680e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 16:24:26.912313 20532 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ab5a0a0f32c249e1a04422a4b14b680e [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:24:26.912492 20532 raft_consensus.cc:686] T 00000000000000000000000000000000 P ab5a0a0f32c249e1a04422a4b14b680e [term 1 LEADER]: Becoming Leader. State: Replica: ab5a0a0f32c249e1a04422a4b14b680e, State: Running, Role: LEADER
I20260809 16:24:26.912532 20524 sys_catalog.cc:564] T 00000000000000000000000000000000 P ab5a0a0f32c249e1a04422a4b14b680e [sys.catalog]: configured and running, proceeding with master startup.
I20260809 16:24:26.912652 20532 consensus_queue.cc:237] T 00000000000000000000000000000000 P ab5a0a0f32c249e1a04422a4b14b680e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab5a0a0f32c249e1a04422a4b14b680e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44935 } }
I20260809 16:24:26.913601 20534 sys_catalog.cc:455] T 00000000000000000000000000000000 P ab5a0a0f32c249e1a04422a4b14b680e [sys.catalog]: SysCatalogTable state changed. Reason: New leader ab5a0a0f32c249e1a04422a4b14b680e. Latest consensus state: current_term: 1 leader_uuid: "ab5a0a0f32c249e1a04422a4b14b680e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab5a0a0f32c249e1a04422a4b14b680e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44935 } } }
I20260809 16:24:26.913697 20534 sys_catalog.cc:458] T 00000000000000000000000000000000 P ab5a0a0f32c249e1a04422a4b14b680e [sys.catalog]: This master's current role is: LEADER
I20260809 16:24:26.914057 20535 sys_catalog.cc:455] T 00000000000000000000000000000000 P ab5a0a0f32c249e1a04422a4b14b680e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ab5a0a0f32c249e1a04422a4b14b680e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab5a0a0f32c249e1a04422a4b14b680e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44935 } } }
I20260809 16:24:26.914135 20535 sys_catalog.cc:458] T 00000000000000000000000000000000 P ab5a0a0f32c249e1a04422a4b14b680e [sys.catalog]: This master's current role is: LEADER
W20260809 16:24:26.914701 20551 catalog_manager.cc:1558] T 00000000000000000000000000000000 P ab5a0a0f32c249e1a04422a4b14b680e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 16:24:26.914775 20551 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 16:24:26.914867 20545 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 16:24:26.915032 20545 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 16:24:26.916616 20545 catalog_manager.cc:1347] Generated new cluster ID: b6e0ada8a0d7475a893a88ef36a2e4a7
I20260809 16:24:26.916669 20545 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 09 16:24:26 dist-test-slave-s11t krb5kdc[20403](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292666, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260809 16:24:26.923662 20531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:26.912055 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51725 (local address 127.0.0.1:44935)
0809 16:24:26.912278 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0809 16:24:26.912288 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 16:24:26.912324 (+ 36us) server_negotiation.cc:407] Connection header received
0809 16:24:26.912442 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:24:26.912449 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:24:26.912502 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:24:26.912707 (+ 205us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:24:26.913573 (+ 866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:24:26.915071 (+ 1498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:24:26.916056 (+ 985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:24:26.916664 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:24:26.916823 (+ 159us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:24:26.919914 (+ 3091us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:24:26.919918 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:24:26.919918 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:24:26.919929 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:24:26.921907 (+ 1978us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:24:26.922367 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:24:26.922369 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:24:26.922370 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:24:26.922491 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:24:26.922769 (+ 278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:24:26.922770 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:24:26.922770 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:24:26.923056 (+ 286us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:24:26.923203 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0809 16:24:26.923354 (+ 151us) server_negotiation.cc:299] Negotiation successful
0809 16:24:26.923458 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":158,"thread_start_us":98,"threads_started":1}
I20260809 16:24:26.925153 21710 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 16:24:26.933841 20531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:26.926563 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46391 (local address 127.0.0.1:44935)
0809 16:24:26.926596 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0809 16:24:26.926598 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:24:26.926721 (+ 123us) server_negotiation.cc:407] Connection header received
0809 16:24:26.926779 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:24:26.926780 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:24:26.926788 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:24:26.926844 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:24:26.927461 (+ 617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:24:26.928193 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:24:26.929238 (+ 1045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:24:26.929554 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:24:26.929617 (+ 63us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:24:26.931006 (+ 1389us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:24:26.931007 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:24:26.931008 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:24:26.931014 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:24:26.932571 (+ 1557us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:24:26.933019 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:24:26.933020 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:24:26.933021 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:24:26.933074 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:24:26.933312 (+ 238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:24:26.933313 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:24:26.933313 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:24:26.933436 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:24:26.933534 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0809 16:24:26.933696 (+ 162us) server_negotiation.cc:299] Negotiation successful
0809 16:24:26.933759 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260809 16:24:26.952204 20545 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 16:24:26.952824 20545 catalog_manager.cc:1504] Loading token signing keys...
I20260809 16:24:26.959329 20545 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ab5a0a0f32c249e1a04422a4b14b680e: Generated new TSK 0
I20260809 16:24:26.959551 20545 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 16:24:26.967059 21710 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrNqUCn/build/release/bin/kudu with pid 20419
W20260809 16:24:26.979907 20399 connection.cc:535] client connection to 127.0.0.1:44935 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:44935): Connection reset by peer (error 104)
2026-08-09T16:24:26Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (1570 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786292417722232-21710-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 16:24:27 dist-test-slave-s11t krb5kdc[20568](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 16:24:27 dist-test-slave-s11t krb5kdc[20568](info): set up 2 sockets
Aug 09 16:24:27 dist-test-slave-s11t krb5kdc[20568](info): commencing operation
krb5kdc: starting...
W20260809 16:24:28.212661 21710 mini_kdc.cc:121] Time spent starting KDC: real 1.219s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786292417722232-21710-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786292417722232-21710-0/krb5kdc/test-user.keytab.
Aug 09 16:24:28 dist-test-slave-s11t krb5kdc[20568](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292668, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T16:24:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T16:24:28Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786292417722232-21710-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786292417722232-21710-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 16:24:28.340399 21710 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrNqUCn/build/release/bin/kudu
/tmp/dist-test-taskrNqUCn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786292417722232-21710-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786292417722232-21710-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786292417722232-21710-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786292417722232-21710-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34723
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786292417722232-21710-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38439
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34723
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786292417722232-21710-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786292417722232-21710-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786292417722232-21710-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:24:28.424948 20585 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:24:28.425195 20585 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:24:28.425221 20585 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:24:28.426404 20585 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 16:24:28.426446 20585 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:24:28.426461 20585 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:24:28.426474 20585 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 16:24:28.426532 20585 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 16:24:28.428012 20585 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38439
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786292417722232-21710-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786292417722232-21710-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34723
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786292417722232-21710-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34723
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786292417722232-21710-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20585
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786292417722232-21710-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:24:28.428267 20585 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:24:28.428516 20585 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:24:28.433403 20594 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:24:28.433530 20591 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:24:28.433384 20590 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:24:28.433708 20585 server_base.cc:1034] running on GCE node
I20260809 16:24:28.433933 20585 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:24:28.434211 20585 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:24:28.435384 20585 hybrid_clock.cc:648] HybridClock initialized: now 1786292668435374 us; error 45 us; skew 500 ppm
Aug 09 16:24:28 dist-test-slave-s11t krb5kdc[20568](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292668, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:24:28.437989 20585 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 16:24:28.438447 20585 webserver.cc:460] Webserver started at http://127.0.0.1:45709/ using document root <none> and password file <none>
I20260809 16:24:28.438643 20585 fs_manager.cc:359] Metadata directory not provided
I20260809 16:24:28.438683 20585 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:24:28.438772 20585 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:24:28.439783 20585 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786292417722232-21710-0/minicluster-data/master-0/data/instance:
uuid: "42efa6c7a3eb47d389c6907a2be6c957"
format_stamp: "Formatted at 2026-08-09 16:24:28 on dist-test-slave-s11t"
server_key: "af26cdada709fbe98f5cf6d3d5f41405"
server_key_iv: "da49f6042169fdb858c5c56260631006"
server_key_version: "encryptionkey@0"
I20260809 16:24:28.440083 20585 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786292417722232-21710-0/minicluster-data/master-0/wal/instance:
uuid: "42efa6c7a3eb47d389c6907a2be6c957"
format_stamp: "Formatted at 2026-08-09 16:24:28 on dist-test-slave-s11t"
server_key: "af26cdada709fbe98f5cf6d3d5f41405"
server_key_iv: "da49f6042169fdb858c5c56260631006"
server_key_version: "encryptionkey@0"
I20260809 16:24:28.441434 20585 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 16:24:28.442099 20603 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:24:28.442282 20585 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 16:24:28.442373 20585 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786292417722232-21710-0/minicluster-data/master-0/data,/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786292417722232-21710-0/minicluster-data/master-0/wal
uuid: "42efa6c7a3eb47d389c6907a2be6c957"
format_stamp: "Formatted at 2026-08-09 16:24:28 on dist-test-slave-s11t"
server_key: "af26cdada709fbe98f5cf6d3d5f41405"
server_key_iv: "da49f6042169fdb858c5c56260631006"
server_key_version: "encryptionkey@0"
I20260809 16:24:28.442435 20585 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786292417722232-21710-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786292417722232-21710-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786292417722232-21710-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:24:28.452113 20585 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:24:28.453840 20585 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:24:28.453948 20585 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:24:28.458144 20585 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34723
I20260809 16:24:28.458204 20683 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34723 every 8 connection(s)
I20260809 16:24:28.458669 20585 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786292417722232-21710-0/minicluster-data/master-0/data/info.pb
I20260809 16:24:28.459573 20685 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:24:28.461854 20685 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 42efa6c7a3eb47d389c6907a2be6c957: Bootstrap starting.
I20260809 16:24:28.462399 20685 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 42efa6c7a3eb47d389c6907a2be6c957: Neither blocks nor log segments found. Creating new log.
I20260809 16:24:28.462637 20685 log.cc:824] T 00000000000000000000000000000000 P 42efa6c7a3eb47d389c6907a2be6c957: Log is configured to *not* fsync() on all Append() calls
I20260809 16:24:28.463315 20685 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 42efa6c7a3eb47d389c6907a2be6c957: No bootstrap required, opened a new log
I20260809 16:24:28.464927 20685 raft_consensus.cc:348] T 00000000000000000000000000000000 P 42efa6c7a3eb47d389c6907a2be6c957 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42efa6c7a3eb47d389c6907a2be6c957" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34723 } }
I20260809 16:24:28.465062 20685 raft_consensus.cc:374] T 00000000000000000000000000000000 P 42efa6c7a3eb47d389c6907a2be6c957 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:24:28.465085 20685 raft_consensus.cc:729] T 00000000000000000000000000000000 P 42efa6c7a3eb47d389c6907a2be6c957 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 42efa6c7a3eb47d389c6907a2be6c957, State: Initialized, Role: FOLLOWER
I20260809 16:24:28.465211 20685 consensus_queue.cc:260] T 00000000000000000000000000000000 P 42efa6c7a3eb47d389c6907a2be6c957 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42efa6c7a3eb47d389c6907a2be6c957" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34723 } }
I20260809 16:24:28.465292 20685 raft_consensus.cc:388] T 00000000000000000000000000000000 P 42efa6c7a3eb47d389c6907a2be6c957 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 16:24:28.465317 20685 raft_consensus.cc:482] T 00000000000000000000000000000000 P 42efa6c7a3eb47d389c6907a2be6c957 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 16:24:28.465343 20685 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 42efa6c7a3eb47d389c6907a2be6c957 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:24:28.465741 21710 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrNqUCn/build/release/bin/kudu as pid 20585
I20260809 16:24:28.465837 21710 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786292417722232-21710-0/minicluster-data/master-0/wal/instance
I20260809 16:24:28.465860 20685 raft_consensus.cc:504] T 00000000000000000000000000000000 P 42efa6c7a3eb47d389c6907a2be6c957 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42efa6c7a3eb47d389c6907a2be6c957" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34723 } }
I20260809 16:24:28.465989 20685 leader_election.cc:302] T 00000000000000000000000000000000 P 42efa6c7a3eb47d389c6907a2be6c957 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 42efa6c7a3eb47d389c6907a2be6c957; no voters:
I20260809 16:24:28.466094 21710 external_mini_cluster.cc:1428] Setting key 850ce7878d23d1c3a576dcf9ffde3e2f
I20260809 16:24:28.466173 20685 leader_election.cc:288] T 00000000000000000000000000000000 P 42efa6c7a3eb47d389c6907a2be6c957 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 16:24:28.466223 20691 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 42efa6c7a3eb47d389c6907a2be6c957 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:24:28.466389 20691 raft_consensus.cc:686] T 00000000000000000000000000000000 P 42efa6c7a3eb47d389c6907a2be6c957 [term 1 LEADER]: Becoming Leader. State: Replica: 42efa6c7a3eb47d389c6907a2be6c957, State: Running, Role: LEADER
I20260809 16:24:28.466445 20685 sys_catalog.cc:564] T 00000000000000000000000000000000 P 42efa6c7a3eb47d389c6907a2be6c957 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 16:24:28.466511 20691 consensus_queue.cc:237] T 00000000000000000000000000000000 P 42efa6c7a3eb47d389c6907a2be6c957 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42efa6c7a3eb47d389c6907a2be6c957" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34723 } }
I20260809 16:24:28.466840 20691 sys_catalog.cc:455] T 00000000000000000000000000000000 P 42efa6c7a3eb47d389c6907a2be6c957 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 42efa6c7a3eb47d389c6907a2be6c957. Latest consensus state: current_term: 1 leader_uuid: "42efa6c7a3eb47d389c6907a2be6c957" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42efa6c7a3eb47d389c6907a2be6c957" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34723 } } }
I20260809 16:24:28.466958 20691 sys_catalog.cc:458] T 00000000000000000000000000000000 P 42efa6c7a3eb47d389c6907a2be6c957 [sys.catalog]: This master's current role is: LEADER
I20260809 16:24:28.467042 20694 sys_catalog.cc:455] T 00000000000000000000000000000000 P 42efa6c7a3eb47d389c6907a2be6c957 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "42efa6c7a3eb47d389c6907a2be6c957" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42efa6c7a3eb47d389c6907a2be6c957" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34723 } } }
I20260809 16:24:28.467157 20694 sys_catalog.cc:458] T 00000000000000000000000000000000 P 42efa6c7a3eb47d389c6907a2be6c957 [sys.catalog]: This master's current role is: LEADER
W20260809 16:24:28.468292 20723 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 42efa6c7a3eb47d389c6907a2be6c957: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 16:24:28.468357 20723 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 16:24:28.468869 20706 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 16:24:28.469058 20706 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 16:24:28.470461 20706 catalog_manager.cc:1347] Generated new cluster ID: f6baac991ace48a980a6fa9f9c895ea1
I20260809 16:24:28.470515 20706 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 16:24:28.473508 20699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:28.466775 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50777 (local address 127.0.0.1:34723)
0809 16:24:28.468009 (+ 1234us) server_negotiation.cc:206] Beginning negotiation
0809 16:24:28.468020 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 16:24:28.468054 (+ 34us) server_negotiation.cc:407] Connection header received
0809 16:24:28.468176 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:24:28.468183 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:24:28.468240 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:24:28.468660 (+ 420us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:24:28.468775 (+ 115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:24:28.470140 (+ 1365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:24:28.471293 (+ 1153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:24:28.471924 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:24:28.472041 (+ 117us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:24:28.472774 (+ 733us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:24:28.472778 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:24:28.472779 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 16:24:28.472997 (+ 218us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:24:28.473039 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 16:24:28.473053 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:24:28.473153 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0809 16:24:28.473251 (+ 98us) server_negotiation.cc:299] Negotiation successful
0809 16:24:28.473284 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1170,"thread_start_us":69,"threads_started":1}
I20260809 16:24:28.474471 21710 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 16:24:28.479444 20699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:28.475881 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60997 (local address 127.0.0.1:34723)
0809 16:24:28.475940 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0809 16:24:28.475942 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:24:28.476150 (+ 208us) server_negotiation.cc:407] Connection header received
0809 16:24:28.476178 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:24:28.476179 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:24:28.476189 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:24:28.476259 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:24:28.476385 (+ 126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:24:28.477160 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:24:28.478138 (+ 978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:24:28.478440 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:24:28.478516 (+ 76us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:24:28.479185 (+ 669us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:24:28.479187 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:24:28.479187 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 16:24:28.479191 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:24:28.479206 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 16:24:28.479210 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:24:28.479270 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0809 16:24:28.479350 (+ 80us) server_negotiation.cc:299] Negotiation successful
0809 16:24:28.479363 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260809 16:24:28.493418 20706 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 16:24:28.493950 20706 catalog_manager.cc:1504] Loading token signing keys...
I20260809 16:24:28.497821 20706 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 42efa6c7a3eb47d389c6907a2be6c957: Generated new TSK 0
I20260809 16:24:28.498018 20706 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 16:24:28.505286 21710 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrNqUCn/build/release/bin/kudu with pid 20585
W20260809 16:24:28.516084 20566 connection.cc:535] client connection to 127.0.0.1:34723 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:34723): Connection reset by peer (error 104)
2026-08-09T16:24:28Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1536 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786292417722232-21710-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 16:24:28 dist-test-slave-s11t krb5kdc[20737](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 16:24:28 dist-test-slave-s11t krb5kdc[20737](info): set up 2 sockets
Aug 09 16:24:28 dist-test-slave-s11t krb5kdc[20737](info): commencing operation
krb5kdc: starting...
W20260809 16:24:29.739902 21710 mini_kdc.cc:121] Time spent starting KDC: real 1.210s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786292417722232-21710-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786292417722232-21710-0/krb5kdc/test-user.keytab.
Aug 09 16:24:29 dist-test-slave-s11t krb5kdc[20737](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292669, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T16:24:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T16:24:29Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786292417722232-21710-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786292417722232-21710-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 16:24:29.863438 21710 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrNqUCn/build/release/bin/kudu
/tmp/dist-test-taskrNqUCn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786292417722232-21710-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786292417722232-21710-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786292417722232-21710-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786292417722232-21710-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38133
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786292417722232-21710-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33017
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38133
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786292417722232-21710-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786292417722232-21710-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786292417722232-21710-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:24:29.949410 20751 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:24:29.949607 20751 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:24:29.949630 20751 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:24:29.950824 20751 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 16:24:29.950868 20751 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:24:29.950882 20751 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:24:29.950894 20751 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 16:24:29.950906 20751 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 16:24:29.952437 20751 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33017
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786292417722232-21710-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786292417722232-21710-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38133
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786292417722232-21710-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38133
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786292417722232-21710-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20751
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786292417722232-21710-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:24:29.952708 20751 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:24:29.952951 20751 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:24:29.958046 20763 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:24:29.958029 20751 server_base.cc:1034] running on GCE node
W20260809 16:24:29.958027 20761 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:24:29.958026 20759 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:24:29.958611 20751 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:24:29.958945 20751 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:24:29.960165 20751 hybrid_clock.cc:648] HybridClock initialized: now 1786292669960150 us; error 30 us; skew 500 ppm
Aug 09 16:24:29 dist-test-slave-s11t krb5kdc[20737](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292669, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:24:29.962816 20751 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 16:24:29.963306 20751 webserver.cc:460] Webserver started at http://127.0.0.1:33925/ using document root <none> and password file <none>
I20260809 16:24:29.963546 20751 fs_manager.cc:359] Metadata directory not provided
I20260809 16:24:29.963614 20751 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:24:29.963773 20751 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:24:29.964941 20751 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786292417722232-21710-0/minicluster-data/master-0/data/instance:
uuid: "2c08e45a44d845caabeaed5fcd705af8"
format_stamp: "Formatted at 2026-08-09 16:24:29 on dist-test-slave-s11t"
server_key: "29a81d03021cf13275428bc4489fa9aa"
server_key_iv: "59f31045edb5920888af92b0908d8261"
server_key_version: "encryptionkey@0"
I20260809 16:24:29.965312 20751 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786292417722232-21710-0/minicluster-data/master-0/wal/instance:
uuid: "2c08e45a44d845caabeaed5fcd705af8"
format_stamp: "Formatted at 2026-08-09 16:24:29 on dist-test-slave-s11t"
server_key: "29a81d03021cf13275428bc4489fa9aa"
server_key_iv: "59f31045edb5920888af92b0908d8261"
server_key_version: "encryptionkey@0"
I20260809 16:24:29.966665 20751 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:24:29.967543 20773 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:24:29.967720 20751 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:24:29.967823 20751 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786292417722232-21710-0/minicluster-data/master-0/data,/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786292417722232-21710-0/minicluster-data/master-0/wal
uuid: "2c08e45a44d845caabeaed5fcd705af8"
format_stamp: "Formatted at 2026-08-09 16:24:29 on dist-test-slave-s11t"
server_key: "29a81d03021cf13275428bc4489fa9aa"
server_key_iv: "59f31045edb5920888af92b0908d8261"
server_key_version: "encryptionkey@0"
I20260809 16:24:29.967911 20751 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786292417722232-21710-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786292417722232-21710-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786292417722232-21710-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:24:29.971323 20751 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:24:29.973289 20751 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:24:29.973444 20751 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:24:29.977725 20751 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38133
I20260809 16:24:29.977784 20854 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38133 every 8 connection(s)
I20260809 16:24:29.978113 20751 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786292417722232-21710-0/minicluster-data/master-0/data/info.pb
I20260809 16:24:29.978650 21710 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrNqUCn/build/release/bin/kudu as pid 20751
I20260809 16:24:29.978791 21710 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786292417722232-21710-0/minicluster-data/master-0/wal/instance
I20260809 16:24:29.979101 21710 external_mini_cluster.cc:1428] Setting key 038237292836db185f68a1ee62b58380
I20260809 16:24:29.979471 20856 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:24:29.981992 20859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:29.979665 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47895 (local address 127.0.0.1:38133)
0809 16:24:29.979883 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0809 16:24:29.979892 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 16:24:29.979919 (+ 27us) server_negotiation.cc:407] Connection header received
0809 16:24:29.980560 (+ 641us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:24:29.980567 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:24:29.980592 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:24:29.980837 (+ 245us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:24:29.981522 (+ 685us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:24:29.981525 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:24:29.981526 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 16:24:29.981530 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:24:29.981583 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 16:24:29.981595 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:24:29.981692 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0809 16:24:29.981770 (+ 78us) server_negotiation.cc:299] Negotiation successful
0809 16:24:29.981815 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":157,"thread_start_us":92,"threads_started":1}
I20260809 16:24:29.982965 20856 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2c08e45a44d845caabeaed5fcd705af8: Bootstrap starting.
I20260809 16:24:29.983079 21710 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 16:24:29.983595 20856 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2c08e45a44d845caabeaed5fcd705af8: Neither blocks nor log segments found. Creating new log.
I20260809 16:24:29.983891 20856 log.cc:824] T 00000000000000000000000000000000 P 2c08e45a44d845caabeaed5fcd705af8: Log is configured to *not* fsync() on all Append() calls
I20260809 16:24:29.984797 20856 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2c08e45a44d845caabeaed5fcd705af8: No bootstrap required, opened a new log
I20260809 16:24:29.985752 20859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:29.984613 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59415 (local address 127.0.0.1:38133)
0809 16:24:29.984649 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0809 16:24:29.984651 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:24:29.984812 (+ 161us) server_negotiation.cc:407] Connection header received
0809 16:24:29.984906 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:24:29.984907 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:24:29.984912 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:24:29.984961 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:24:29.985520 (+ 559us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:24:29.985521 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:24:29.985522 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 16:24:29.985526 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:24:29.985539 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 16:24:29.985543 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:24:29.985606 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0809 16:24:29.985671 (+ 65us) server_negotiation.cc:299] Negotiation successful
0809 16:24:29.985685 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260809 16:24:29.986970 20856 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2c08e45a44d845caabeaed5fcd705af8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2c08e45a44d845caabeaed5fcd705af8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38133 } }
I20260809 16:24:29.987100 20856 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2c08e45a44d845caabeaed5fcd705af8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:24:29.987124 20856 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2c08e45a44d845caabeaed5fcd705af8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c08e45a44d845caabeaed5fcd705af8, State: Initialized, Role: FOLLOWER
I20260809 16:24:29.987238 20856 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2c08e45a44d845caabeaed5fcd705af8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2c08e45a44d845caabeaed5fcd705af8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38133 } }
I20260809 16:24:29.987318 20856 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2c08e45a44d845caabeaed5fcd705af8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 16:24:29.987347 20856 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2c08e45a44d845caabeaed5fcd705af8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 16:24:29.987375 20856 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2c08e45a44d845caabeaed5fcd705af8 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:24:29.988004 20856 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2c08e45a44d845caabeaed5fcd705af8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2c08e45a44d845caabeaed5fcd705af8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38133 } }
I20260809 16:24:29.988090 20856 leader_election.cc:302] T 00000000000000000000000000000000 P 2c08e45a44d845caabeaed5fcd705af8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2c08e45a44d845caabeaed5fcd705af8; no voters:
I20260809 16:24:29.988204 20856 leader_election.cc:288] T 00000000000000000000000000000000 P 2c08e45a44d845caabeaed5fcd705af8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 16:24:29.988257 20869 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2c08e45a44d845caabeaed5fcd705af8 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:24:29.988436 20869 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2c08e45a44d845caabeaed5fcd705af8 [term 1 LEADER]: Becoming Leader. State: Replica: 2c08e45a44d845caabeaed5fcd705af8, State: Running, Role: LEADER
I20260809 16:24:29.988466 20856 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2c08e45a44d845caabeaed5fcd705af8 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 16:24:29.988597 20869 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2c08e45a44d845caabeaed5fcd705af8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2c08e45a44d845caabeaed5fcd705af8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38133 } }
I20260809 16:24:29.989061 20871 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2c08e45a44d845caabeaed5fcd705af8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2c08e45a44d845caabeaed5fcd705af8. Latest consensus state: current_term: 1 leader_uuid: "2c08e45a44d845caabeaed5fcd705af8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2c08e45a44d845caabeaed5fcd705af8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38133 } } }
I20260809 16:24:29.989053 20870 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2c08e45a44d845caabeaed5fcd705af8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2c08e45a44d845caabeaed5fcd705af8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2c08e45a44d845caabeaed5fcd705af8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38133 } } }
I20260809 16:24:29.989243 20870 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2c08e45a44d845caabeaed5fcd705af8 [sys.catalog]: This master's current role is: LEADER
I20260809 16:24:29.989606 20871 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2c08e45a44d845caabeaed5fcd705af8 [sys.catalog]: This master's current role is: LEADER
I20260809 16:24:29.989715 20882 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 16:24:29.990156 20882 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 16:24:29.991741 20882 catalog_manager.cc:1347] Generated new cluster ID: 2afd59206ef24e40b7f0da00a99a58f4
I20260809 16:24:29.991801 20882 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 16:24:30.006325 20882 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 16:24:30.006541 20882 catalog_manager.cc:1504] Loading token signing keys...
I20260809 16:24:30.013077 20882 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2c08e45a44d845caabeaed5fcd705af8: Generated new TSK 0
I20260809 16:24:30.013288 20882 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 16:24:30.022055 21710 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrNqUCn/build/release/bin/kudu with pid 20751
2026-08-09T16:24:30Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1519 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786292417722232-21710-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 16:24:30 dist-test-slave-s11t krb5kdc[20901](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 16:24:30 dist-test-slave-s11t krb5kdc[20901](info): set up 2 sockets
Aug 09 16:24:30 dist-test-slave-s11t krb5kdc[20901](info): commencing operation
krb5kdc: starting...
W20260809 16:24:31.310774 21710 mini_kdc.cc:121] Time spent starting KDC: real 1.261s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786292417722232-21710-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786292417722232-21710-0/krb5kdc/test-user.keytab.
Aug 09 16:24:31 dist-test-slave-s11t krb5kdc[20901](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292671, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T16:24:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T16:24:31Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786292417722232-21710-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786292417722232-21710-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 16:24:31.433616 21710 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrNqUCn/build/release/bin/kudu
/tmp/dist-test-taskrNqUCn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786292417722232-21710-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786292417722232-21710-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786292417722232-21710-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786292417722232-21710-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36835
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786292417722232-21710-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34469
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36835
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786292417722232-21710-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786292417722232-21710-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786292417722232-21710-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:24:31.515319 20921 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:24:31.515528 20921 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:24:31.515551 20921 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:24:31.516726 20921 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 16:24:31.516767 20921 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:24:31.516781 20921 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:24:31.516793 20921 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 16:24:31.516805 20921 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 16:24:31.518263 20921 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34469
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786292417722232-21710-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786292417722232-21710-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36835
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786292417722232-21710-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36835
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786292417722232-21710-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20921
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786292417722232-21710-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:24:31.518510 20921 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:24:31.518746 20921 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 16:24:31.523682 20921 server_base.cc:1034] running on GCE node
W20260809 16:24:31.523643 20937 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:24:31.523679 20941 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:24:31.523643 20936 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:24:31.524150 20921 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:24:31.524461 20921 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:24:31.525607 20921 hybrid_clock.cc:648] HybridClock initialized: now 1786292671525594 us; error 31 us; skew 500 ppm
Aug 09 16:24:31 dist-test-slave-s11t krb5kdc[20901](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292671, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:24:31.528059 20921 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 16:24:31.528501 20921 webserver.cc:460] Webserver started at http://127.0.0.1:42737/ using document root <none> and password file <none>
I20260809 16:24:31.528791 20921 fs_manager.cc:359] Metadata directory not provided
I20260809 16:24:31.528869 20921 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:24:31.529029 20921 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:24:31.530079 20921 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786292417722232-21710-0/minicluster-data/master-0/data/instance:
uuid: "ba5e854cad014e9da0b213aef73983a5"
format_stamp: "Formatted at 2026-08-09 16:24:31 on dist-test-slave-s11t"
server_key: "77c9e86bec93c950e28c1bae67f59cf5"
server_key_iv: "f6de391f5cf9efd513acb0106c6da35a"
server_key_version: "encryptionkey@0"
I20260809 16:24:31.530449 20921 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786292417722232-21710-0/minicluster-data/master-0/wal/instance:
uuid: "ba5e854cad014e9da0b213aef73983a5"
format_stamp: "Formatted at 2026-08-09 16:24:31 on dist-test-slave-s11t"
server_key: "77c9e86bec93c950e28c1bae67f59cf5"
server_key_iv: "f6de391f5cf9efd513acb0106c6da35a"
server_key_version: "encryptionkey@0"
I20260809 16:24:31.531832 20921 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 16:24:31.532658 20951 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:24:31.533203 20921 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 16:24:31.533301 20921 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786292417722232-21710-0/minicluster-data/master-0/data,/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786292417722232-21710-0/minicluster-data/master-0/wal
uuid: "ba5e854cad014e9da0b213aef73983a5"
format_stamp: "Formatted at 2026-08-09 16:24:31 on dist-test-slave-s11t"
server_key: "77c9e86bec93c950e28c1bae67f59cf5"
server_key_iv: "f6de391f5cf9efd513acb0106c6da35a"
server_key_version: "encryptionkey@0"
I20260809 16:24:31.533394 20921 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786292417722232-21710-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786292417722232-21710-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786292417722232-21710-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:24:31.545919 20921 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:24:31.547837 20921 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:24:31.547999 20921 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:24:31.552171 20921 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36835
I20260809 16:24:31.552222 21032 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36835 every 8 connection(s)
I20260809 16:24:31.552588 20921 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786292417722232-21710-0/minicluster-data/master-0/data/info.pb
I20260809 16:24:31.553545 21034 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:24:31.555850 21034 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ba5e854cad014e9da0b213aef73983a5: Bootstrap starting.
I20260809 16:24:31.556429 21034 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ba5e854cad014e9da0b213aef73983a5: Neither blocks nor log segments found. Creating new log.
I20260809 16:24:31.556788 21034 log.cc:824] T 00000000000000000000000000000000 P ba5e854cad014e9da0b213aef73983a5: Log is configured to *not* fsync() on all Append() calls
I20260809 16:24:31.557499 21034 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ba5e854cad014e9da0b213aef73983a5: No bootstrap required, opened a new log
I20260809 16:24:31.558734 21710 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrNqUCn/build/release/bin/kudu as pid 20921
I20260809 16:24:31.558851 21710 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786292417722232-21710-0/minicluster-data/master-0/wal/instance
I20260809 16:24:31.559052 21710 external_mini_cluster.cc:1428] Setting key 5de3c241c6b9e37ac8a631844ddfb6df
I20260809 16:24:31.559747 21034 raft_consensus.cc:348] T 00000000000000000000000000000000 P ba5e854cad014e9da0b213aef73983a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba5e854cad014e9da0b213aef73983a5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36835 } }
I20260809 16:24:31.559947 21034 raft_consensus.cc:374] T 00000000000000000000000000000000 P ba5e854cad014e9da0b213aef73983a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:24:31.559990 21034 raft_consensus.cc:729] T 00000000000000000000000000000000 P ba5e854cad014e9da0b213aef73983a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ba5e854cad014e9da0b213aef73983a5, State: Initialized, Role: FOLLOWER
I20260809 16:24:31.560114 21034 consensus_queue.cc:260] T 00000000000000000000000000000000 P ba5e854cad014e9da0b213aef73983a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba5e854cad014e9da0b213aef73983a5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36835 } }
I20260809 16:24:31.560190 21034 raft_consensus.cc:388] T 00000000000000000000000000000000 P ba5e854cad014e9da0b213aef73983a5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 16:24:31.560235 21034 raft_consensus.cc:482] T 00000000000000000000000000000000 P ba5e854cad014e9da0b213aef73983a5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 16:24:31.560289 21034 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ba5e854cad014e9da0b213aef73983a5 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:24:31.561553 21034 raft_consensus.cc:504] T 00000000000000000000000000000000 P ba5e854cad014e9da0b213aef73983a5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba5e854cad014e9da0b213aef73983a5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36835 } }
I20260809 16:24:31.561668 21034 leader_election.cc:302] T 00000000000000000000000000000000 P ba5e854cad014e9da0b213aef73983a5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ba5e854cad014e9da0b213aef73983a5; no voters:
I20260809 16:24:31.561821 21034 leader_election.cc:288] T 00000000000000000000000000000000 P ba5e854cad014e9da0b213aef73983a5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 16:24:31.561899 21042 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ba5e854cad014e9da0b213aef73983a5 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:24:31.562079 21042 raft_consensus.cc:686] T 00000000000000000000000000000000 P ba5e854cad014e9da0b213aef73983a5 [term 1 LEADER]: Becoming Leader. State: Replica: ba5e854cad014e9da0b213aef73983a5, State: Running, Role: LEADER
I20260809 16:24:31.562089 21034 sys_catalog.cc:564] T 00000000000000000000000000000000 P ba5e854cad014e9da0b213aef73983a5 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 16:24:31.562189 21042 consensus_queue.cc:237] T 00000000000000000000000000000000 P ba5e854cad014e9da0b213aef73983a5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba5e854cad014e9da0b213aef73983a5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36835 } }
I20260809 16:24:31.562582 21042 sys_catalog.cc:455] T 00000000000000000000000000000000 P ba5e854cad014e9da0b213aef73983a5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ba5e854cad014e9da0b213aef73983a5. Latest consensus state: current_term: 1 leader_uuid: "ba5e854cad014e9da0b213aef73983a5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba5e854cad014e9da0b213aef73983a5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36835 } } }
I20260809 16:24:31.562662 21042 sys_catalog.cc:458] T 00000000000000000000000000000000 P ba5e854cad014e9da0b213aef73983a5 [sys.catalog]: This master's current role is: LEADER
I20260809 16:24:31.562840 21044 sys_catalog.cc:455] T 00000000000000000000000000000000 P ba5e854cad014e9da0b213aef73983a5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ba5e854cad014e9da0b213aef73983a5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba5e854cad014e9da0b213aef73983a5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36835 } } }
I20260809 16:24:31.562928 21044 sys_catalog.cc:458] T 00000000000000000000000000000000 P ba5e854cad014e9da0b213aef73983a5 [sys.catalog]: This master's current role is: LEADER
I20260809 16:24:31.563045 21054 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 16:24:31.563774 21054 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 09 16:24:31 dist-test-slave-s11t krb5kdc[20901](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292671, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260809 16:24:31.569159 21054 catalog_manager.cc:1347] Generated new cluster ID: ddde9c60ed2442bb9974d9d88c7d2603
I20260809 16:24:31.569212 21054 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 16:24:31.571352 21040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:31.559772 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56775 (local address 127.0.0.1:36835)
0809 16:24:31.559934 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0809 16:24:31.559944 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 16:24:31.559976 (+ 32us) server_negotiation.cc:407] Connection header received
0809 16:24:31.560089 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:24:31.560096 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:24:31.560149 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:24:31.561215 (+ 1066us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:24:31.561343 (+ 128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:24:31.562765 (+ 1422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:24:31.563963 (+ 1198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:24:31.564561 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:24:31.565016 (+ 455us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:24:31.567243 (+ 2227us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:24:31.567247 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:24:31.567247 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:24:31.567258 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:24:31.569158 (+ 1900us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:24:31.569580 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:24:31.569583 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:24:31.569584 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:24:31.570155 (+ 571us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:24:31.570410 (+ 255us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:24:31.570411 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:24:31.570411 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:24:31.570708 (+ 297us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:24:31.570856 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0809 16:24:31.570993 (+ 137us) server_negotiation.cc:299] Negotiation successful
0809 16:24:31.571123 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":109,"thread_start_us":74,"threads_started":1}
I20260809 16:24:31.572635 21710 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 16:24:31.585516 21054 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 16:24:31.586210 21054 catalog_manager.cc:1504] Loading token signing keys...
I20260809 16:24:31.588400 21040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:31.576987 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51261 (local address 127.0.0.1:36835)
0809 16:24:31.577027 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0809 16:24:31.577029 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:24:31.580906 (+ 3877us) server_negotiation.cc:407] Connection header received
0809 16:24:31.580961 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:24:31.580963 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:24:31.580974 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:24:31.581021 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:24:31.581717 (+ 696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:24:31.582467 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:24:31.583530 (+ 1063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:24:31.583858 (+ 328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:24:31.583929 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:24:31.585583 (+ 1654us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:24:31.585584 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:24:31.585585 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:24:31.585593 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:24:31.587200 (+ 1607us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:24:31.587603 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:24:31.587604 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:24:31.587605 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:24:31.587663 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:24:31.587866 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:24:31.587867 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:24:31.587867 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:24:31.587987 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:24:31.588073 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0809 16:24:31.588220 (+ 147us) server_negotiation.cc:299] Negotiation successful
0809 16:24:31.588313 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260809 16:24:31.590555 21054 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ba5e854cad014e9da0b213aef73983a5: Generated new TSK 0
I20260809 16:24:31.590705 21054 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 16:24:31.595552 21710 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrNqUCn/build/release/bin/kudu with pid 20921
2026-08-09T16:24:31Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (1580 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786292417722232-21710-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 16:24:31 dist-test-slave-s11t krb5kdc[21073](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 16:24:31 dist-test-slave-s11t krb5kdc[21073](info): set up 2 sockets
Aug 09 16:24:31 dist-test-slave-s11t krb5kdc[21073](info): commencing operation
krb5kdc: starting...
W20260809 16:24:32.842272 21710 mini_kdc.cc:121] Time spent starting KDC: real 1.211s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786292417722232-21710-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786292417722232-21710-0/krb5kdc/test-user.keytab.
Aug 09 16:24:32 dist-test-slave-s11t krb5kdc[21073](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292672, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T16:24:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T16:24:32Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786292417722232-21710-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786292417722232-21710-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 16:24:32.967352 21710 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrNqUCn/build/release/bin/kudu
/tmp/dist-test-taskrNqUCn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786292417722232-21710-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786292417722232-21710-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786292417722232-21710-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786292417722232-21710-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39785
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786292417722232-21710-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:46473
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39785
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786292417722232-21710-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786292417722232-21710-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786292417722232-21710-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:24:33.053102 21090 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:24:33.053313 21090 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:24:33.053334 21090 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:24:33.054489 21090 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 16:24:33.054534 21090 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:24:33.054548 21090 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:24:33.054601 21090 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 16:24:33.054629 21090 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 16:24:33.056115 21090 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:46473
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786292417722232-21710-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786292417722232-21710-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39785
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786292417722232-21710-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39785
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786292417722232-21710-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21090
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786292417722232-21710-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:24:33.056370 21090 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:24:33.056604 21090 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:24:33.061594 21098 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:24:33.061584 21100 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:24:33.061584 21096 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:24:33.061862 21090 server_base.cc:1034] running on GCE node
I20260809 16:24:33.062130 21090 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:24:33.062481 21090 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:24:33.063651 21090 hybrid_clock.cc:648] HybridClock initialized: now 1786292673063647 us; error 36 us; skew 500 ppm
Aug 09 16:24:33 dist-test-slave-s11t krb5kdc[21073](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292673, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:24:33.076210 21090 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 16:24:33.076799 21090 webserver.cc:460] Webserver started at http://127.0.0.1:36101/ using document root <none> and password file <none>
I20260809 16:24:33.077042 21090 fs_manager.cc:359] Metadata directory not provided
I20260809 16:24:33.077114 21090 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:24:33.077281 21090 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:24:33.078361 21090 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786292417722232-21710-0/minicluster-data/master-0/data/instance:
uuid: "875105ccd4714c019373d8f1608c8b5f"
format_stamp: "Formatted at 2026-08-09 16:24:33 on dist-test-slave-s11t"
server_key: "4bc25b601cdd139a32fe126b7eb98416"
server_key_iv: "2f4ea2e715d826bb700323331efa9613"
server_key_version: "encryptionkey@0"
I20260809 16:24:33.078740 21090 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786292417722232-21710-0/minicluster-data/master-0/wal/instance:
uuid: "875105ccd4714c019373d8f1608c8b5f"
format_stamp: "Formatted at 2026-08-09 16:24:33 on dist-test-slave-s11t"
server_key: "4bc25b601cdd139a32fe126b7eb98416"
server_key_iv: "2f4ea2e715d826bb700323331efa9613"
server_key_version: "encryptionkey@0"
I20260809 16:24:33.080343 21090 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 16:24:33.081449 21112 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:24:33.081669 21090 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:24:33.081769 21090 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786292417722232-21710-0/minicluster-data/master-0/data,/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786292417722232-21710-0/minicluster-data/master-0/wal
uuid: "875105ccd4714c019373d8f1608c8b5f"
format_stamp: "Formatted at 2026-08-09 16:24:33 on dist-test-slave-s11t"
server_key: "4bc25b601cdd139a32fe126b7eb98416"
server_key_iv: "2f4ea2e715d826bb700323331efa9613"
server_key_version: "encryptionkey@0"
I20260809 16:24:33.081853 21090 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786292417722232-21710-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786292417722232-21710-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786292417722232-21710-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:24:33.101810 21090 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:24:33.104182 21090 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:24:33.104354 21090 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:24:33.108616 21090 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39785
I20260809 16:24:33.108652 21194 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39785 every 8 connection(s)
I20260809 16:24:33.109064 21090 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786292417722232-21710-0/minicluster-data/master-0/data/info.pb
I20260809 16:24:33.109934 21195 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:24:33.112200 21195 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 875105ccd4714c019373d8f1608c8b5f: Bootstrap starting.
I20260809 16:24:33.112824 21195 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 875105ccd4714c019373d8f1608c8b5f: Neither blocks nor log segments found. Creating new log.
I20260809 16:24:33.113140 21195 log.cc:824] T 00000000000000000000000000000000 P 875105ccd4714c019373d8f1608c8b5f: Log is configured to *not* fsync() on all Append() calls
I20260809 16:24:33.113163 21710 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrNqUCn/build/release/bin/kudu as pid 21090
I20260809 16:24:33.113286 21710 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786292417722232-21710-0/minicluster-data/master-0/wal/instance
I20260809 16:24:33.113507 21710 external_mini_cluster.cc:1428] Setting key 61e8714a36f739b018d438415493ae3c
I20260809 16:24:33.113943 21195 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 875105ccd4714c019373d8f1608c8b5f: No bootstrap required, opened a new log
I20260809 16:24:33.115595 21195 raft_consensus.cc:348] T 00000000000000000000000000000000 P 875105ccd4714c019373d8f1608c8b5f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "875105ccd4714c019373d8f1608c8b5f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39785 } }
I20260809 16:24:33.115753 21195 raft_consensus.cc:374] T 00000000000000000000000000000000 P 875105ccd4714c019373d8f1608c8b5f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:24:33.115800 21195 raft_consensus.cc:729] T 00000000000000000000000000000000 P 875105ccd4714c019373d8f1608c8b5f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 875105ccd4714c019373d8f1608c8b5f, State: Initialized, Role: FOLLOWER
I20260809 16:24:33.115933 21195 consensus_queue.cc:260] T 00000000000000000000000000000000 P 875105ccd4714c019373d8f1608c8b5f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "875105ccd4714c019373d8f1608c8b5f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39785 } }
I20260809 16:24:33.116037 21195 raft_consensus.cc:388] T 00000000000000000000000000000000 P 875105ccd4714c019373d8f1608c8b5f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 16:24:33.116070 21195 raft_consensus.cc:482] T 00000000000000000000000000000000 P 875105ccd4714c019373d8f1608c8b5f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 16:24:33.116120 21195 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 875105ccd4714c019373d8f1608c8b5f [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:24:33.116896 21195 raft_consensus.cc:504] T 00000000000000000000000000000000 P 875105ccd4714c019373d8f1608c8b5f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "875105ccd4714c019373d8f1608c8b5f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39785 } }
I20260809 16:24:33.117017 21195 leader_election.cc:302] T 00000000000000000000000000000000 P 875105ccd4714c019373d8f1608c8b5f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 875105ccd4714c019373d8f1608c8b5f; no voters:
I20260809 16:24:33.117172 21195 leader_election.cc:288] T 00000000000000000000000000000000 P 875105ccd4714c019373d8f1608c8b5f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 16:24:33.117221 21200 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 875105ccd4714c019373d8f1608c8b5f [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:24:33.117398 21195 sys_catalog.cc:564] T 00000000000000000000000000000000 P 875105ccd4714c019373d8f1608c8b5f [sys.catalog]: configured and running, proceeding with master startup.
I20260809 16:24:33.117417 21200 raft_consensus.cc:686] T 00000000000000000000000000000000 P 875105ccd4714c019373d8f1608c8b5f [term 1 LEADER]: Becoming Leader. State: Replica: 875105ccd4714c019373d8f1608c8b5f, State: Running, Role: LEADER
I20260809 16:24:33.117501 21200 consensus_queue.cc:237] T 00000000000000000000000000000000 P 875105ccd4714c019373d8f1608c8b5f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "875105ccd4714c019373d8f1608c8b5f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39785 } }
I20260809 16:24:33.117859 21202 sys_catalog.cc:455] T 00000000000000000000000000000000 P 875105ccd4714c019373d8f1608c8b5f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "875105ccd4714c019373d8f1608c8b5f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "875105ccd4714c019373d8f1608c8b5f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39785 } } }
I20260809 16:24:33.117951 21202 sys_catalog.cc:458] T 00000000000000000000000000000000 P 875105ccd4714c019373d8f1608c8b5f [sys.catalog]: This master's current role is: LEADER
I20260809 16:24:33.117842 21200 sys_catalog.cc:455] T 00000000000000000000000000000000 P 875105ccd4714c019373d8f1608c8b5f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 875105ccd4714c019373d8f1608c8b5f. Latest consensus state: current_term: 1 leader_uuid: "875105ccd4714c019373d8f1608c8b5f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "875105ccd4714c019373d8f1608c8b5f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39785 } } }
I20260809 16:24:33.118152 21200 sys_catalog.cc:458] T 00000000000000000000000000000000 P 875105ccd4714c019373d8f1608c8b5f [sys.catalog]: This master's current role is: LEADER
I20260809 16:24:33.118784 21211 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 16:24:33.119293 21211 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 16:24:33.121227 21211 catalog_manager.cc:1347] Generated new cluster ID: 2c1e2a4654db4a86b28e2286d1a85e27
I20260809 16:24:33.121255 21199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:33.114027 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35879 (local address 127.0.0.1:39785)
0809 16:24:33.114222 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0809 16:24:33.114232 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 16:24:33.114265 (+ 33us) server_negotiation.cc:407] Connection header received
0809 16:24:33.115580 (+ 1315us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:24:33.115586 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:24:33.115633 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:24:33.115773 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:24:33.116036 (+ 263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:24:33.117856 (+ 1820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:24:33.119069 (+ 1213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:24:33.119656 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:24:33.119768 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:24:33.120505 (+ 737us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:24:33.120509 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:24:33.120509 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 16:24:33.120756 (+ 247us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:24:33.120796 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 16:24:33.120812 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:24:33.120886 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0809 16:24:33.120981 (+ 95us) server_negotiation.cc:299] Negotiation successful
0809 16:24:33.121045 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":137,"thread_start_us":81,"threads_started":1}
I20260809 16:24:33.121307 21211 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 16:24:33.122192 21710 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 16:24:33.126995 21199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:33.123607 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36023 (local address 127.0.0.1:39785)
0809 16:24:33.123642 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0809 16:24:33.123644 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:24:33.123844 (+ 200us) server_negotiation.cc:407] Connection header received
0809 16:24:33.123887 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:24:33.123888 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:24:33.123916 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:24:33.123972 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:24:33.124114 (+ 142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:24:33.124832 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:24:33.125775 (+ 943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:24:33.126074 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:24:33.126155 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:24:33.126757 (+ 602us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:24:33.126758 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:24:33.126759 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 16:24:33.126764 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:24:33.126779 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 16:24:33.126783 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:24:33.126843 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0809 16:24:33.126899 (+ 56us) server_negotiation.cc:299] Negotiation successful
0809 16:24:33.126922 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260809 16:24:33.135936 21211 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 16:24:33.136469 21211 catalog_manager.cc:1504] Loading token signing keys...
I20260809 16:24:33.150554 21211 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 875105ccd4714c019373d8f1608c8b5f: Generated new TSK 0
I20260809 16:24:33.150830 21211 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 16:24:33.161111 21710 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrNqUCn/build/release/bin/kudu with pid 21090
2026-08-09T16:24:33Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1557 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786292417722232-21710-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 16:24:33 dist-test-slave-s11t krb5kdc[21230](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 16:24:33 dist-test-slave-s11t krb5kdc[21230](info): set up 2 sockets
Aug 09 16:24:33 dist-test-slave-s11t krb5kdc[21230](info): commencing operation
krb5kdc: starting...
W20260809 16:24:34.405809 21710 mini_kdc.cc:121] Time spent starting KDC: real 1.217s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786292417722232-21710-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786292417722232-21710-0/krb5kdc/test-user.keytab.
Aug 09 16:24:34 dist-test-slave-s11t krb5kdc[21230](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292674, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T16:24:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T16:24:34Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786292417722232-21710-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786292417722232-21710-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 16:24:34.527561 21710 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrNqUCn/build/release/bin/kudu
/tmp/dist-test-taskrNqUCn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786292417722232-21710-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786292417722232-21710-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786292417722232-21710-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786292417722232-21710-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36451
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786292417722232-21710-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40731
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36451
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786292417722232-21710-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786292417722232-21710-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786292417722232-21710-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:24:34.609664 21250 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:24:34.609902 21250 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:24:34.609937 21250 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:24:34.611055 21250 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 16:24:34.611123 21250 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:24:34.611147 21250 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:24:34.611169 21250 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 16:24:34.611197 21250 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 16:24:34.612674 21250 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40731
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786292417722232-21710-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786292417722232-21710-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36451
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786292417722232-21710-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36451
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786292417722232-21710-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21250
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786292417722232-21710-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:24:34.612963 21250 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:24:34.613188 21250 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:24:34.618317 21257 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:24:34.618359 21259 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:24:34.618319 21263 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:24:34.618631 21250 server_base.cc:1034] running on GCE node
I20260809 16:24:34.618881 21250 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:24:34.619202 21250 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:24:34.620374 21250 hybrid_clock.cc:648] HybridClock initialized: now 1786292674620352 us; error 36 us; skew 500 ppm
Aug 09 16:24:34 dist-test-slave-s11t krb5kdc[21230](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292674, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:24:34.623020 21250 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 16:24:34.623485 21250 webserver.cc:460] Webserver started at http://127.0.0.1:43299/ using document root <none> and password file <none>
I20260809 16:24:34.623737 21250 fs_manager.cc:359] Metadata directory not provided
I20260809 16:24:34.623807 21250 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:24:34.623960 21250 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:24:34.625072 21250 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786292417722232-21710-0/minicluster-data/master-0/data/instance:
uuid: "22e19cd4afc248f5b4ded2922301ef01"
format_stamp: "Formatted at 2026-08-09 16:24:34 on dist-test-slave-s11t"
server_key: "24f14bf0714b2ba20bc0bb16abdbe2d3"
server_key_iv: "a91eef236a880ba2000a4ff263053656"
server_key_version: "encryptionkey@0"
I20260809 16:24:34.625437 21250 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786292417722232-21710-0/minicluster-data/master-0/wal/instance:
uuid: "22e19cd4afc248f5b4ded2922301ef01"
format_stamp: "Formatted at 2026-08-09 16:24:34 on dist-test-slave-s11t"
server_key: "24f14bf0714b2ba20bc0bb16abdbe2d3"
server_key_iv: "a91eef236a880ba2000a4ff263053656"
server_key_version: "encryptionkey@0"
I20260809 16:24:34.626783 21250 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.001s
I20260809 16:24:34.627588 21273 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:24:34.627782 21250 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:24:34.627888 21250 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786292417722232-21710-0/minicluster-data/master-0/data,/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786292417722232-21710-0/minicluster-data/master-0/wal
uuid: "22e19cd4afc248f5b4ded2922301ef01"
format_stamp: "Formatted at 2026-08-09 16:24:34 on dist-test-slave-s11t"
server_key: "24f14bf0714b2ba20bc0bb16abdbe2d3"
server_key_iv: "a91eef236a880ba2000a4ff263053656"
server_key_version: "encryptionkey@0"
I20260809 16:24:34.627985 21250 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786292417722232-21710-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786292417722232-21710-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786292417722232-21710-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:24:34.631366 21250 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:24:34.633725 21250 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:24:34.633885 21250 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:24:34.638267 21250 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36451
I20260809 16:24:34.638300 21350 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36451 every 8 connection(s)
I20260809 16:24:34.638625 21250 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786292417722232-21710-0/minicluster-data/master-0/data/info.pb
I20260809 16:24:34.639605 21354 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:24:34.641923 21354 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 22e19cd4afc248f5b4ded2922301ef01: Bootstrap starting.
I20260809 16:24:34.642273 21710 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrNqUCn/build/release/bin/kudu as pid 21250
I20260809 16:24:34.642428 21710 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786292417722232-21710-0/minicluster-data/master-0/wal/instance
I20260809 16:24:34.642659 21710 external_mini_cluster.cc:1428] Setting key 0edb61da5b61018821ea913c81f1c8f9
I20260809 16:24:34.642680 21354 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 22e19cd4afc248f5b4ded2922301ef01: Neither blocks nor log segments found. Creating new log.
I20260809 16:24:34.643023 21354 log.cc:824] T 00000000000000000000000000000000 P 22e19cd4afc248f5b4ded2922301ef01: Log is configured to *not* fsync() on all Append() calls
I20260809 16:24:34.643836 21354 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 22e19cd4afc248f5b4ded2922301ef01: No bootstrap required, opened a new log
I20260809 16:24:34.645296 21359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:34.643154 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39523 (local address 127.0.0.1:36451)
0809 16:24:34.643335 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0809 16:24:34.643344 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 16:24:34.643380 (+ 36us) server_negotiation.cc:407] Connection header received
0809 16:24:34.643992 (+ 612us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:24:34.644002 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:24:34.644035 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:24:34.644183 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:24:34.644871 (+ 688us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:24:34.644874 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:24:34.644875 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 16:24:34.644879 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:24:34.644925 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 16:24:34.644939 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:24:34.645031 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0809 16:24:34.645069 (+ 38us) server_negotiation.cc:299] Negotiation successful
0809 16:24:34.645108 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":121,"thread_start_us":73,"threads_started":1}
I20260809 16:24:34.645776 21354 raft_consensus.cc:348] T 00000000000000000000000000000000 P 22e19cd4afc248f5b4ded2922301ef01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "22e19cd4afc248f5b4ded2922301ef01" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36451 } }
I20260809 16:24:34.645973 21354 raft_consensus.cc:374] T 00000000000000000000000000000000 P 22e19cd4afc248f5b4ded2922301ef01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:24:34.646021 21354 raft_consensus.cc:729] T 00000000000000000000000000000000 P 22e19cd4afc248f5b4ded2922301ef01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 22e19cd4afc248f5b4ded2922301ef01, State: Initialized, Role: FOLLOWER
I20260809 16:24:34.646155 21354 consensus_queue.cc:260] T 00000000000000000000000000000000 P 22e19cd4afc248f5b4ded2922301ef01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "22e19cd4afc248f5b4ded2922301ef01" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36451 } }
I20260809 16:24:34.646270 21354 raft_consensus.cc:388] T 00000000000000000000000000000000 P 22e19cd4afc248f5b4ded2922301ef01 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 16:24:34.646319 21354 raft_consensus.cc:482] T 00000000000000000000000000000000 P 22e19cd4afc248f5b4ded2922301ef01 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 16:24:34.646378 21354 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 22e19cd4afc248f5b4ded2922301ef01 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:24:34.646615 21710 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 16:24:34.647084 21354 raft_consensus.cc:504] T 00000000000000000000000000000000 P 22e19cd4afc248f5b4ded2922301ef01 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "22e19cd4afc248f5b4ded2922301ef01" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36451 } }
I20260809 16:24:34.647207 21354 leader_election.cc:302] T 00000000000000000000000000000000 P 22e19cd4afc248f5b4ded2922301ef01 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 22e19cd4afc248f5b4ded2922301ef01; no voters:
I20260809 16:24:34.647384 21354 leader_election.cc:288] T 00000000000000000000000000000000 P 22e19cd4afc248f5b4ded2922301ef01 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 16:24:34.647418 21361 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 22e19cd4afc248f5b4ded2922301ef01 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:24:34.647579 21361 raft_consensus.cc:686] T 00000000000000000000000000000000 P 22e19cd4afc248f5b4ded2922301ef01 [term 1 LEADER]: Becoming Leader. State: Replica: 22e19cd4afc248f5b4ded2922301ef01, State: Running, Role: LEADER
I20260809 16:24:34.647632 21354 sys_catalog.cc:564] T 00000000000000000000000000000000 P 22e19cd4afc248f5b4ded2922301ef01 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 16:24:34.647672 21361 consensus_queue.cc:237] T 00000000000000000000000000000000 P 22e19cd4afc248f5b4ded2922301ef01 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "22e19cd4afc248f5b4ded2922301ef01" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36451 } }
I20260809 16:24:34.648003 21361 sys_catalog.cc:455] T 00000000000000000000000000000000 P 22e19cd4afc248f5b4ded2922301ef01 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 22e19cd4afc248f5b4ded2922301ef01. Latest consensus state: current_term: 1 leader_uuid: "22e19cd4afc248f5b4ded2922301ef01" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "22e19cd4afc248f5b4ded2922301ef01" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36451 } } }
I20260809 16:24:34.648074 21361 sys_catalog.cc:458] T 00000000000000000000000000000000 P 22e19cd4afc248f5b4ded2922301ef01 [sys.catalog]: This master's current role is: LEADER
I20260809 16:24:34.648234 21363 sys_catalog.cc:455] T 00000000000000000000000000000000 P 22e19cd4afc248f5b4ded2922301ef01 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "22e19cd4afc248f5b4ded2922301ef01" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "22e19cd4afc248f5b4ded2922301ef01" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36451 } } }
I20260809 16:24:34.648303 21363 sys_catalog.cc:458] T 00000000000000000000000000000000 P 22e19cd4afc248f5b4ded2922301ef01 [sys.catalog]: This master's current role is: LEADER
I20260809 16:24:34.649492 21374 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 16:24:34.650007 21374 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 16:24:34.650678 21359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:34.648181 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54213 (local address 127.0.0.1:36451)
0809 16:24:34.648222 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0809 16:24:34.648224 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:24:34.648481 (+ 257us) server_negotiation.cc:407] Connection header received
0809 16:24:34.648548 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:24:34.648549 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:24:34.648553 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:24:34.648630 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:24:34.649230 (+ 600us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:24:34.649231 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:24:34.649232 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 16:24:34.649237 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:24:34.649251 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 16:24:34.649255 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:24:34.649300 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0809 16:24:34.650605 (+ 1305us) server_negotiation.cc:299] Negotiation successful
0809 16:24:34.650619 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260809 16:24:34.651454 21374 catalog_manager.cc:1347] Generated new cluster ID: 46f1fa9c8bb649f599ef1adffe0089d4
I20260809 16:24:34.651515 21374 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 16:24:34.657994 21374 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 16:24:34.658146 21374 catalog_manager.cc:1504] Loading token signing keys...
I20260809 16:24:34.667213 21374 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 22e19cd4afc248f5b4ded2922301ef01: Generated new TSK 0
I20260809 16:24:34.667410 21374 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 16:24:34.670215 21710 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrNqUCn/build/release/bin/kudu with pid 21250
2026-08-09T16:24:34Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1507 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786292417722232-21710-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 16:24:34 dist-test-slave-s11t krb5kdc[21397](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 16:24:34 dist-test-slave-s11t krb5kdc[21397](info): set up 2 sockets
Aug 09 16:24:34 dist-test-slave-s11t krb5kdc[21397](info): commencing operation
krb5kdc: starting...
W20260809 16:24:35.932389 21710 mini_kdc.cc:121] Time spent starting KDC: real 1.236s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786292417722232-21710-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786292417722232-21710-0/krb5kdc/test-user.keytab.
Aug 09 16:24:36 dist-test-slave-s11t krb5kdc[21397](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292676, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T16:24:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T16:24:36Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786292417722232-21710-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786292417722232-21710-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 16:24:36.055817 21710 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrNqUCn/build/release/bin/kudu
/tmp/dist-test-taskrNqUCn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786292417722232-21710-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786292417722232-21710-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786292417722232-21710-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786292417722232-21710-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33505
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786292417722232-21710-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37421
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33505
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786292417722232-21710-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786292417722232-21710-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786292417722232-21710-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:24:36.140178 21419 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:24:36.140427 21419 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:24:36.140465 21419 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:24:36.141645 21419 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 16:24:36.141718 21419 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:24:36.141755 21419 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:24:36.141798 21419 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 16:24:36.141824 21419 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 16:24:36.143319 21419 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37421
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786292417722232-21710-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786292417722232-21710-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33505
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786292417722232-21710-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33505
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786292417722232-21710-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21419
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786292417722232-21710-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:24:36.143604 21419 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:24:36.143842 21419 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:24:36.149031 21425 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:24:36.149072 21429 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:24:36.149035 21426 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:24:36.149209 21419 server_base.cc:1034] running on GCE node
I20260809 16:24:36.149572 21419 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:24:36.149888 21419 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:24:36.151062 21419 hybrid_clock.cc:648] HybridClock initialized: now 1786292676151048 us; error 45 us; skew 500 ppm
Aug 09 16:24:36 dist-test-slave-s11t krb5kdc[21397](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292676, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:24:36.153617 21419 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 16:24:36.154034 21419 webserver.cc:460] Webserver started at http://127.0.0.1:42665/ using document root <none> and password file <none>
I20260809 16:24:36.154284 21419 fs_manager.cc:359] Metadata directory not provided
I20260809 16:24:36.154357 21419 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:24:36.154510 21419 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:24:36.155553 21419 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786292417722232-21710-0/minicluster-data/master-0/data/instance:
uuid: "44eaa5a58d1a408ab41197498b72b547"
format_stamp: "Formatted at 2026-08-09 16:24:36 on dist-test-slave-s11t"
server_key: "095d336fd0d20223928e3a8a59a30637"
server_key_iv: "a22eb7d1560375ff14c1ff029f432ab8"
server_key_version: "encryptionkey@0"
I20260809 16:24:36.155920 21419 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786292417722232-21710-0/minicluster-data/master-0/wal/instance:
uuid: "44eaa5a58d1a408ab41197498b72b547"
format_stamp: "Formatted at 2026-08-09 16:24:36 on dist-test-slave-s11t"
server_key: "095d336fd0d20223928e3a8a59a30637"
server_key_iv: "a22eb7d1560375ff14c1ff029f432ab8"
server_key_version: "encryptionkey@0"
I20260809 16:24:36.157411 21419 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 16:24:36.158216 21440 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:24:36.158407 21419 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:24:36.158514 21419 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786292417722232-21710-0/minicluster-data/master-0/data,/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786292417722232-21710-0/minicluster-data/master-0/wal
uuid: "44eaa5a58d1a408ab41197498b72b547"
format_stamp: "Formatted at 2026-08-09 16:24:36 on dist-test-slave-s11t"
server_key: "095d336fd0d20223928e3a8a59a30637"
server_key_iv: "a22eb7d1560375ff14c1ff029f432ab8"
server_key_version: "encryptionkey@0"
I20260809 16:24:36.158589 21419 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786292417722232-21710-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786292417722232-21710-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786292417722232-21710-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:24:36.168006 21419 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:24:36.169924 21419 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:24:36.170080 21419 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:24:36.174383 21419 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33505
I20260809 16:24:36.174450 21525 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33505 every 8 connection(s)
I20260809 16:24:36.174782 21419 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786292417722232-21710-0/minicluster-data/master-0/data/info.pb
I20260809 16:24:36.175650 21529 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:24:36.177904 21529 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 44eaa5a58d1a408ab41197498b72b547: Bootstrap starting.
I20260809 16:24:36.178469 21529 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 44eaa5a58d1a408ab41197498b72b547: Neither blocks nor log segments found. Creating new log.
I20260809 16:24:36.178700 21529 log.cc:824] T 00000000000000000000000000000000 P 44eaa5a58d1a408ab41197498b72b547: Log is configured to *not* fsync() on all Append() calls
I20260809 16:24:36.179356 21529 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 44eaa5a58d1a408ab41197498b72b547: No bootstrap required, opened a new log
I20260809 16:24:36.180960 21529 raft_consensus.cc:348] T 00000000000000000000000000000000 P 44eaa5a58d1a408ab41197498b72b547 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44eaa5a58d1a408ab41197498b72b547" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33505 } }
I20260809 16:24:36.181093 21529 raft_consensus.cc:374] T 00000000000000000000000000000000 P 44eaa5a58d1a408ab41197498b72b547 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:24:36.181115 21529 raft_consensus.cc:729] T 00000000000000000000000000000000 P 44eaa5a58d1a408ab41197498b72b547 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 44eaa5a58d1a408ab41197498b72b547, State: Initialized, Role: FOLLOWER
I20260809 16:24:36.181268 21529 consensus_queue.cc:260] T 00000000000000000000000000000000 P 44eaa5a58d1a408ab41197498b72b547 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44eaa5a58d1a408ab41197498b72b547" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33505 } }
I20260809 16:24:36.181349 21710 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrNqUCn/build/release/bin/kudu as pid 21419
I20260809 16:24:36.181362 21529 raft_consensus.cc:388] T 00000000000000000000000000000000 P 44eaa5a58d1a408ab41197498b72b547 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 16:24:36.181411 21529 raft_consensus.cc:482] T 00000000000000000000000000000000 P 44eaa5a58d1a408ab41197498b72b547 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 16:24:36.181456 21710 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786292417722232-21710-0/minicluster-data/master-0/wal/instance
I20260809 16:24:36.181468 21529 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 44eaa5a58d1a408ab41197498b72b547 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:24:36.181666 21710 external_mini_cluster.cc:1428] Setting key 23771945faf82809b8a410a073892c1d
I20260809 16:24:36.182224 21529 raft_consensus.cc:504] T 00000000000000000000000000000000 P 44eaa5a58d1a408ab41197498b72b547 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44eaa5a58d1a408ab41197498b72b547" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33505 } }
I20260809 16:24:36.182348 21529 leader_election.cc:302] T 00000000000000000000000000000000 P 44eaa5a58d1a408ab41197498b72b547 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 44eaa5a58d1a408ab41197498b72b547; no voters:
I20260809 16:24:36.182530 21529 leader_election.cc:288] T 00000000000000000000000000000000 P 44eaa5a58d1a408ab41197498b72b547 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 16:24:36.182564 21539 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 44eaa5a58d1a408ab41197498b72b547 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:24:36.182767 21539 raft_consensus.cc:686] T 00000000000000000000000000000000 P 44eaa5a58d1a408ab41197498b72b547 [term 1 LEADER]: Becoming Leader. State: Replica: 44eaa5a58d1a408ab41197498b72b547, State: Running, Role: LEADER
I20260809 16:24:36.182786 21529 sys_catalog.cc:564] T 00000000000000000000000000000000 P 44eaa5a58d1a408ab41197498b72b547 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 16:24:36.182874 21539 consensus_queue.cc:237] T 00000000000000000000000000000000 P 44eaa5a58d1a408ab41197498b72b547 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44eaa5a58d1a408ab41197498b72b547" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33505 } }
I20260809 16:24:36.183179 21540 sys_catalog.cc:455] T 00000000000000000000000000000000 P 44eaa5a58d1a408ab41197498b72b547 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "44eaa5a58d1a408ab41197498b72b547" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44eaa5a58d1a408ab41197498b72b547" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33505 } } }
I20260809 16:24:36.183288 21540 sys_catalog.cc:458] T 00000000000000000000000000000000 P 44eaa5a58d1a408ab41197498b72b547 [sys.catalog]: This master's current role is: LEADER
I20260809 16:24:36.183481 21544 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 16:24:36.183455 21541 sys_catalog.cc:455] T 00000000000000000000000000000000 P 44eaa5a58d1a408ab41197498b72b547 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 44eaa5a58d1a408ab41197498b72b547. Latest consensus state: current_term: 1 leader_uuid: "44eaa5a58d1a408ab41197498b72b547" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44eaa5a58d1a408ab41197498b72b547" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33505 } } }
I20260809 16:24:36.183593 21541 sys_catalog.cc:458] T 00000000000000000000000000000000 P 44eaa5a58d1a408ab41197498b72b547 [sys.catalog]: This master's current role is: LEADER
I20260809 16:24:36.184161 21544 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 16:24:36.186120 21544 catalog_manager.cc:1347] Generated new cluster ID: be8078d52b9d468ea6d80ca4a3550caa
I20260809 16:24:36.186184 21544 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 09 16:24:36 dist-test-slave-s11t krb5kdc[21397](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292676, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260809 16:24:36.192811 21544 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 16:24:36.193285 21544 catalog_manager.cc:1504] Loading token signing keys...
I20260809 16:24:36.194770 21536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:36.182241 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.76:39363 (local address 127.0.0.1:33505)
0809 16:24:36.182463 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0809 16:24:36.182473 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 16:24:36.182512 (+ 39us) server_negotiation.cc:407] Connection header received
0809 16:24:36.182631 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:24:36.182639 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:24:36.182712 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:24:36.183758 (+ 1046us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:24:36.183915 (+ 157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:24:36.185292 (+ 1377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:24:36.186351 (+ 1059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:24:36.186959 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:24:36.187060 (+ 101us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:24:36.189955 (+ 2895us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:24:36.189959 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:24:36.189959 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:24:36.189974 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:24:36.192705 (+ 2731us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:24:36.193181 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:24:36.193184 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:24:36.193184 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:24:36.193293 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:24:36.193544 (+ 251us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:24:36.193545 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:24:36.193546 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:24:36.193844 (+ 298us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:24:36.193996 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0809 16:24:36.194435 (+ 439us) server_negotiation.cc:299] Negotiation successful
0809 16:24:36.194541 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":174,"thread_start_us":104,"threads_started":1}
I20260809 16:24:36.195741 21710 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 16:24:36.208245 21536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:36.197267 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.76:53689 (local address 127.0.0.1:33505)
0809 16:24:36.197302 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0809 16:24:36.197304 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:24:36.200856 (+ 3552us) server_negotiation.cc:407] Connection header received
0809 16:24:36.200891 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:24:36.200892 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:24:36.200902 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:24:36.200973 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:24:36.201544 (+ 571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:24:36.202343 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:24:36.203482 (+ 1139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:24:36.203881 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:24:36.203960 (+ 79us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:24:36.205331 (+ 1371us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:24:36.205333 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:24:36.205333 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:24:36.205342 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:24:36.206955 (+ 1613us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:24:36.207366 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:24:36.207367 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:24:36.207367 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:24:36.207433 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:24:36.207663 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:24:36.207664 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:24:36.207664 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:24:36.207830 (+ 166us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:24:36.207937 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0809 16:24:36.208085 (+ 148us) server_negotiation.cc:299] Negotiation successful
0809 16:24:36.208152 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260809 16:24:36.223150 21544 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 44eaa5a58d1a408ab41197498b72b547: Generated new TSK 0
I20260809 16:24:36.223362 21544 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 16:24:36.227331 21710 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrNqUCn/build/release/bin/kudu with pid 21419
W20260809 16:24:36.240429 21395 connection.cc:535] client connection to 127.0.0.1:33505 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:33505): Connection reset by peer (error 104)
2026-08-09T16:24:36Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (1558 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786292417722232-21710-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 16:24:36 dist-test-slave-s11t krb5kdc[21582](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 16:24:36 dist-test-slave-s11t krb5kdc[21582](info): set up 2 sockets
Aug 09 16:24:36 dist-test-slave-s11t krb5kdc[21582](info): commencing operation
krb5kdc: starting...
W20260809 16:24:37.554467 21710 mini_kdc.cc:121] Time spent starting KDC: real 1.300s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786292417722232-21710-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786292417722232-21710-0/krb5kdc/test-user.keytab.
Aug 09 16:24:37 dist-test-slave-s11t krb5kdc[21582](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292677, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T16:24:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T16:24:37Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786292417722232-21710-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786292417722232-21710-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 16:24:37.683506 21710 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrNqUCn/build/release/bin/kudu
/tmp/dist-test-taskrNqUCn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786292417722232-21710-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786292417722232-21710-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786292417722232-21710-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786292417722232-21710-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45629
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786292417722232-21710-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42781
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45629
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786292417722232-21710-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786292417722232-21710-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786292417722232-21710-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:24:37.770303 21603 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:24:37.770540 21603 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:24:37.770566 21603 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:24:37.771876 21603 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 16:24:37.771927 21603 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:24:37.771943 21603 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:24:37.771957 21603 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 16:24:37.771971 21603 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 16:24:37.773753 21603 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42781
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786292417722232-21710-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786292417722232-21710-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45629
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786292417722232-21710-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45629
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786292417722232-21710-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21603
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786292417722232-21710-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:24:37.774034 21603 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:24:37.774297 21603 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:24:37.779811 21612 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:24:37.779858 21617 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:24:37.780006 21611 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:24:37.780292 21603 server_base.cc:1034] running on GCE node
I20260809 16:24:37.780570 21603 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:24:37.780916 21603 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:24:37.782086 21603 hybrid_clock.cc:648] HybridClock initialized: now 1786292677782077 us; error 44 us; skew 500 ppm
Aug 09 16:24:37 dist-test-slave-s11t krb5kdc[21582](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292677, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:24:37.784775 21603 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 16:24:37.785238 21603 webserver.cc:460] Webserver started at http://127.0.0.1:40457/ using document root <none> and password file <none>
I20260809 16:24:37.785487 21603 fs_manager.cc:359] Metadata directory not provided
I20260809 16:24:37.785557 21603 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:24:37.785714 21603 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:24:37.786772 21603 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786292417722232-21710-0/minicluster-data/master-0/data/instance:
uuid: "f122ab2d191849a4b58fbcb10a634023"
format_stamp: "Formatted at 2026-08-09 16:24:37 on dist-test-slave-s11t"
server_key: "d8a5cac5593c38f8c78eb95abca098c3"
server_key_iv: "7e39ac9d33df39bf3bb68e3b3d999d9b"
server_key_version: "encryptionkey@0"
I20260809 16:24:37.787168 21603 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786292417722232-21710-0/minicluster-data/master-0/wal/instance:
uuid: "f122ab2d191849a4b58fbcb10a634023"
format_stamp: "Formatted at 2026-08-09 16:24:37 on dist-test-slave-s11t"
server_key: "d8a5cac5593c38f8c78eb95abca098c3"
server_key_iv: "7e39ac9d33df39bf3bb68e3b3d999d9b"
server_key_version: "encryptionkey@0"
I20260809 16:24:37.788537 21603 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 16:24:37.789474 21634 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:24:37.789783 21603 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 16:24:37.789930 21603 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786292417722232-21710-0/minicluster-data/master-0/data,/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786292417722232-21710-0/minicluster-data/master-0/wal
uuid: "f122ab2d191849a4b58fbcb10a634023"
format_stamp: "Formatted at 2026-08-09 16:24:37 on dist-test-slave-s11t"
server_key: "d8a5cac5593c38f8c78eb95abca098c3"
server_key_iv: "7e39ac9d33df39bf3bb68e3b3d999d9b"
server_key_version: "encryptionkey@0"
I20260809 16:24:37.790007 21603 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786292417722232-21710-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786292417722232-21710-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786292417722232-21710-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:24:37.817510 21603 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:24:37.819396 21603 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:24:37.819569 21603 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:24:37.823640 21603 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45629
I20260809 16:24:37.823676 21718 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45629 every 8 connection(s)
I20260809 16:24:37.824038 21603 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786292417722232-21710-0/minicluster-data/master-0/data/info.pb
I20260809 16:24:37.824997 21720 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:24:37.827251 21720 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f122ab2d191849a4b58fbcb10a634023: Bootstrap starting.
I20260809 16:24:37.827842 21720 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f122ab2d191849a4b58fbcb10a634023: Neither blocks nor log segments found. Creating new log.
I20260809 16:24:37.828137 21720 log.cc:824] T 00000000000000000000000000000000 P f122ab2d191849a4b58fbcb10a634023: Log is configured to *not* fsync() on all Append() calls
I20260809 16:24:37.828758 21710 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrNqUCn/build/release/bin/kudu as pid 21603
I20260809 16:24:37.828905 21710 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786292417722232-21710-0/minicluster-data/master-0/wal/instance
I20260809 16:24:37.829005 21720 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f122ab2d191849a4b58fbcb10a634023: No bootstrap required, opened a new log
I20260809 16:24:37.829147 21710 external_mini_cluster.cc:1428] Setting key f28fe0ef731612d2eda49370968ab2e9
I20260809 16:24:37.831632 21720 raft_consensus.cc:348] T 00000000000000000000000000000000 P f122ab2d191849a4b58fbcb10a634023 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f122ab2d191849a4b58fbcb10a634023" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45629 } }
I20260809 16:24:37.831822 21720 raft_consensus.cc:374] T 00000000000000000000000000000000 P f122ab2d191849a4b58fbcb10a634023 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:24:37.831862 21720 raft_consensus.cc:729] T 00000000000000000000000000000000 P f122ab2d191849a4b58fbcb10a634023 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f122ab2d191849a4b58fbcb10a634023, State: Initialized, Role: FOLLOWER
I20260809 16:24:37.831992 21720 consensus_queue.cc:260] T 00000000000000000000000000000000 P f122ab2d191849a4b58fbcb10a634023 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f122ab2d191849a4b58fbcb10a634023" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45629 } }
I20260809 16:24:37.832069 21720 raft_consensus.cc:388] T 00000000000000000000000000000000 P f122ab2d191849a4b58fbcb10a634023 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 16:24:37.832099 21720 raft_consensus.cc:482] T 00000000000000000000000000000000 P f122ab2d191849a4b58fbcb10a634023 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 16:24:37.832137 21720 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f122ab2d191849a4b58fbcb10a634023 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:24:37.832872 21720 raft_consensus.cc:504] T 00000000000000000000000000000000 P f122ab2d191849a4b58fbcb10a634023 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f122ab2d191849a4b58fbcb10a634023" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45629 } }
I20260809 16:24:37.832968 21720 leader_election.cc:302] T 00000000000000000000000000000000 P f122ab2d191849a4b58fbcb10a634023 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f122ab2d191849a4b58fbcb10a634023; no voters:
I20260809 16:24:37.833171 21720 leader_election.cc:288] T 00000000000000000000000000000000 P f122ab2d191849a4b58fbcb10a634023 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 16:24:37.833195 21730 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f122ab2d191849a4b58fbcb10a634023 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:24:37.833307 21730 raft_consensus.cc:686] T 00000000000000000000000000000000 P f122ab2d191849a4b58fbcb10a634023 [term 1 LEADER]: Becoming Leader. State: Replica: f122ab2d191849a4b58fbcb10a634023, State: Running, Role: LEADER
I20260809 16:24:37.833452 21730 consensus_queue.cc:237] T 00000000000000000000000000000000 P f122ab2d191849a4b58fbcb10a634023 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f122ab2d191849a4b58fbcb10a634023" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45629 } }
I20260809 16:24:37.833460 21720 sys_catalog.cc:564] T 00000000000000000000000000000000 P f122ab2d191849a4b58fbcb10a634023 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 16:24:37.833845 21731 sys_catalog.cc:455] T 00000000000000000000000000000000 P f122ab2d191849a4b58fbcb10a634023 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f122ab2d191849a4b58fbcb10a634023" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f122ab2d191849a4b58fbcb10a634023" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45629 } } }
I20260809 16:24:37.833849 21732 sys_catalog.cc:455] T 00000000000000000000000000000000 P f122ab2d191849a4b58fbcb10a634023 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f122ab2d191849a4b58fbcb10a634023. Latest consensus state: current_term: 1 leader_uuid: "f122ab2d191849a4b58fbcb10a634023" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f122ab2d191849a4b58fbcb10a634023" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45629 } } }
I20260809 16:24:37.834004 21731 sys_catalog.cc:458] T 00000000000000000000000000000000 P f122ab2d191849a4b58fbcb10a634023 [sys.catalog]: This master's current role is: LEADER
I20260809 16:24:37.834033 21732 sys_catalog.cc:458] T 00000000000000000000000000000000 P f122ab2d191849a4b58fbcb10a634023 [sys.catalog]: This master's current role is: LEADER
I20260809 16:24:37.834215 21734 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 16:24:37.834766 21734 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 16:24:37.835129 21727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:37.829686 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.76:37899 (local address 127.0.0.1:45629)
0809 16:24:37.829888 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0809 16:24:37.829895 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 16:24:37.829920 (+ 25us) server_negotiation.cc:407] Connection header received
0809 16:24:37.830340 (+ 420us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:24:37.830347 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:24:37.830395 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:24:37.830508 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:24:37.830749 (+ 241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:24:37.831636 (+ 887us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:24:37.832816 (+ 1180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:24:37.833415 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:24:37.833555 (+ 140us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:24:37.834311 (+ 756us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:24:37.834315 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:24:37.834315 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 16:24:37.834544 (+ 229us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:24:37.834585 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 16:24:37.834599 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:24:37.834688 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0809 16:24:37.834882 (+ 194us) server_negotiation.cc:299] Negotiation successful
0809 16:24:37.834913 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":145,"thread_start_us":95,"threads_started":1}
I20260809 16:24:37.836184 21710 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 16:24:37.837563 21734 catalog_manager.cc:1347] Generated new cluster ID: 2088fda4ed7f4f5a85fa02d0c2b825d5
I20260809 16:24:37.837674 21734 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 16:24:37.844182 21727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:37.837684 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.76:58605 (local address 127.0.0.1:45629)
0809 16:24:37.837721 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0809 16:24:37.837723 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:24:37.840861 (+ 3138us) server_negotiation.cc:407] Connection header received
0809 16:24:37.840906 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:24:37.840907 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:24:37.840917 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:24:37.840987 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:24:37.841125 (+ 138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:24:37.841851 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:24:37.842847 (+ 996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:24:37.843191 (+ 344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:24:37.843270 (+ 79us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:24:37.843933 (+ 663us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:24:37.843934 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:24:37.843934 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 16:24:37.843939 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:24:37.843955 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 16:24:37.843960 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:24:37.844026 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0809 16:24:37.844098 (+ 72us) server_negotiation.cc:299] Negotiation successful
0809 16:24:37.844109 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260809 16:24:37.874082 21734 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 16:24:37.874655 21734 catalog_manager.cc:1504] Loading token signing keys...
I20260809 16:24:37.879287 21734 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f122ab2d191849a4b58fbcb10a634023: Generated new TSK 0
I20260809 16:24:37.879469 21734 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 16:24:37.890028 21710 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrNqUCn/build/release/bin/kudu with pid 21603
W20260809 16:24:37.905453 21580 connection.cc:535] client connection to 127.0.0.1:45629 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:45629): Connection reset by peer (error 104)
2026-08-09T16:24:37Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1664 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786292417722232-21710-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 16:24:37 dist-test-slave-s11t krb5kdc[21771](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 16:24:37 dist-test-slave-s11t krb5kdc[21771](info): set up 2 sockets
Aug 09 16:24:37 dist-test-slave-s11t krb5kdc[21771](info): commencing operation
krb5kdc: starting...
W20260809 16:24:39.141129 21710 mini_kdc.cc:121] Time spent starting KDC: real 1.222s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786292417722232-21710-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786292417722232-21710-0/krb5kdc/test-user.keytab.
Aug 09 16:24:39 dist-test-slave-s11t krb5kdc[21771](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292679, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T16:24:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T16:24:39Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786292417722232-21710-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786292417722232-21710-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 16:24:39.264437 21710 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrNqUCn/build/release/bin/kudu
/tmp/dist-test-taskrNqUCn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786292417722232-21710-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786292417722232-21710-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786292417722232-21710-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786292417722232-21710-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38003
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786292417722232-21710-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:46577
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38003
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786292417722232-21710-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786292417722232-21710-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786292417722232-21710-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:24:39.346515 21789 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:24:39.346709 21789 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:24:39.346731 21789 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:24:39.347873 21789 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 16:24:39.347919 21789 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:24:39.347934 21789 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:24:39.347947 21789 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 16:24:39.348006 21789 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 16:24:39.349509 21789 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:46577
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786292417722232-21710-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786292417722232-21710-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38003
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786292417722232-21710-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38003
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786292417722232-21710-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21789
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786292417722232-21710-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:24:39.349740 21789 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:24:39.349988 21789 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:24:39.355136 21805 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:24:39.355111 21808 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:24:39.355122 21804 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:24:39.355386 21789 server_base.cc:1034] running on GCE node
I20260809 16:24:39.355607 21789 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:24:39.355844 21789 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:24:39.357016 21789 hybrid_clock.cc:648] HybridClock initialized: now 1786292679356999 us; error 38 us; skew 500 ppm
Aug 09 16:24:39 dist-test-slave-s11t krb5kdc[21771](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292679, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:24:39.365175 21789 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 16:24:39.365675 21789 webserver.cc:460] Webserver started at http://127.0.0.1:35037/ using document root <none> and password file <none>
I20260809 16:24:39.365924 21789 fs_manager.cc:359] Metadata directory not provided
I20260809 16:24:39.365972 21789 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:24:39.366138 21789 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:24:39.367208 21789 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786292417722232-21710-0/minicluster-data/master-0/data/instance:
uuid: "8b73489a6620464bb75d9c9c0490e957"
format_stamp: "Formatted at 2026-08-09 16:24:39 on dist-test-slave-s11t"
server_key: "24a833e2839bb1b13f666af292aab626"
server_key_iv: "e97656b9714fd528c65c139f367dce54"
server_key_version: "encryptionkey@0"
I20260809 16:24:39.367575 21789 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786292417722232-21710-0/minicluster-data/master-0/wal/instance:
uuid: "8b73489a6620464bb75d9c9c0490e957"
format_stamp: "Formatted at 2026-08-09 16:24:39 on dist-test-slave-s11t"
server_key: "24a833e2839bb1b13f666af292aab626"
server_key_iv: "e97656b9714fd528c65c139f367dce54"
server_key_version: "encryptionkey@0"
I20260809 16:24:39.369027 21789 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 16:24:39.369927 21819 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:24:39.370103 21789 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:24:39.370229 21789 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786292417722232-21710-0/minicluster-data/master-0/data,/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786292417722232-21710-0/minicluster-data/master-0/wal
uuid: "8b73489a6620464bb75d9c9c0490e957"
format_stamp: "Formatted at 2026-08-09 16:24:39 on dist-test-slave-s11t"
server_key: "24a833e2839bb1b13f666af292aab626"
server_key_iv: "e97656b9714fd528c65c139f367dce54"
server_key_version: "encryptionkey@0"
I20260809 16:24:39.370306 21789 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786292417722232-21710-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786292417722232-21710-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786292417722232-21710-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:24:39.373845 21789 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:24:39.375648 21789 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:24:39.375818 21789 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:24:39.380160 21789 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38003
I20260809 16:24:39.380196 21890 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38003 every 8 connection(s)
I20260809 16:24:39.380522 21789 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786292417722232-21710-0/minicluster-data/master-0/data/info.pb
I20260809 16:24:39.381651 21891 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:24:39.383808 21891 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8b73489a6620464bb75d9c9c0490e957: Bootstrap starting.
I20260809 16:24:39.384325 21891 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8b73489a6620464bb75d9c9c0490e957: Neither blocks nor log segments found. Creating new log.
I20260809 16:24:39.384567 21891 log.cc:824] T 00000000000000000000000000000000 P 8b73489a6620464bb75d9c9c0490e957: Log is configured to *not* fsync() on all Append() calls
I20260809 16:24:39.385267 21891 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8b73489a6620464bb75d9c9c0490e957: No bootstrap required, opened a new log
I20260809 16:24:39.386751 21891 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8b73489a6620464bb75d9c9c0490e957 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b73489a6620464bb75d9c9c0490e957" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38003 } }
I20260809 16:24:39.386880 21891 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8b73489a6620464bb75d9c9c0490e957 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:24:39.386902 21891 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8b73489a6620464bb75d9c9c0490e957 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8b73489a6620464bb75d9c9c0490e957, State: Initialized, Role: FOLLOWER
I20260809 16:24:39.387045 21891 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8b73489a6620464bb75d9c9c0490e957 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b73489a6620464bb75d9c9c0490e957" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38003 } }
I20260809 16:24:39.387125 21891 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8b73489a6620464bb75d9c9c0490e957 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 16:24:39.387148 21891 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8b73489a6620464bb75d9c9c0490e957 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 16:24:39.387176 21891 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8b73489a6620464bb75d9c9c0490e957 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:24:39.387653 21891 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8b73489a6620464bb75d9c9c0490e957 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b73489a6620464bb75d9c9c0490e957" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38003 } }
I20260809 16:24:39.387737 21891 leader_election.cc:302] T 00000000000000000000000000000000 P 8b73489a6620464bb75d9c9c0490e957 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8b73489a6620464bb75d9c9c0490e957; no voters:
I20260809 16:24:39.387884 21891 leader_election.cc:288] T 00000000000000000000000000000000 P 8b73489a6620464bb75d9c9c0490e957 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 16:24:39.387923 21894 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8b73489a6620464bb75d9c9c0490e957 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:24:39.388047 21894 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8b73489a6620464bb75d9c9c0490e957 [term 1 LEADER]: Becoming Leader. State: Replica: 8b73489a6620464bb75d9c9c0490e957, State: Running, Role: LEADER
I20260809 16:24:39.388149 21891 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8b73489a6620464bb75d9c9c0490e957 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 16:24:39.388219 21894 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8b73489a6620464bb75d9c9c0490e957 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b73489a6620464bb75d9c9c0490e957" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38003 } }
I20260809 16:24:39.388602 21895 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8b73489a6620464bb75d9c9c0490e957 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8b73489a6620464bb75d9c9c0490e957" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b73489a6620464bb75d9c9c0490e957" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38003 } } }
I20260809 16:24:39.388787 21895 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8b73489a6620464bb75d9c9c0490e957 [sys.catalog]: This master's current role is: LEADER
I20260809 16:24:39.388801 21897 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8b73489a6620464bb75d9c9c0490e957 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8b73489a6620464bb75d9c9c0490e957. Latest consensus state: current_term: 1 leader_uuid: "8b73489a6620464bb75d9c9c0490e957" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b73489a6620464bb75d9c9c0490e957" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38003 } } }
I20260809 16:24:39.389005 21897 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8b73489a6620464bb75d9c9c0490e957 [sys.catalog]: This master's current role is: LEADER
I20260809 16:24:39.389263 21905 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 16:24:39.389585 21710 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrNqUCn/build/release/bin/kudu as pid 21789
I20260809 16:24:39.389721 21710 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786292417722232-21710-0/minicluster-data/master-0/wal/instance
I20260809 16:24:39.389803 21905 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 16:24:39.389963 21710 external_mini_cluster.cc:1428] Setting key 0e8219c8a9b19b9b154c40d8b8809c0c
I20260809 16:24:39.391783 21905 catalog_manager.cc:1347] Generated new cluster ID: c68171f9636640ee87b10a1b31067bb4
I20260809 16:24:39.391844 21905 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 16:24:39.392283 21928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:39.390562 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.76:58673 (local address 127.0.0.1:38003)
0809 16:24:39.390732 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0809 16:24:39.390741 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 16:24:39.390777 (+ 36us) server_negotiation.cc:407] Connection header received
0809 16:24:39.390897 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:24:39.390904 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:24:39.390936 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:24:39.391107 (+ 171us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:24:39.391850 (+ 743us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:24:39.391854 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:24:39.391854 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 16:24:39.391859 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:24:39.391902 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 16:24:39.391928 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:24:39.391998 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0809 16:24:39.392087 (+ 89us) server_negotiation.cc:299] Negotiation successful
0809 16:24:39.392127 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":120,"thread_start_us":69,"threads_started":1}
I20260809 16:24:39.393304 21710 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 16:24:39.396160 21928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:39.394924 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.76:46205 (local address 127.0.0.1:38003)
0809 16:24:39.394977 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0809 16:24:39.394979 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:24:39.395221 (+ 242us) server_negotiation.cc:407] Connection header received
0809 16:24:39.395248 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:24:39.395249 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:24:39.395255 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:24:39.395324 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:24:39.395935 (+ 611us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:24:39.395936 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:24:39.395936 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 16:24:39.395941 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:24:39.395959 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 16:24:39.395964 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:24:39.396034 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0809 16:24:39.396082 (+ 48us) server_negotiation.cc:299] Negotiation successful
0809 16:24:39.396095 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260809 16:24:39.412482 21905 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 16:24:39.412663 21905 catalog_manager.cc:1504] Loading token signing keys...
I20260809 16:24:39.421130 21905 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8b73489a6620464bb75d9c9c0490e957: Generated new TSK 0
I20260809 16:24:39.421320 21905 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 16:24:39.423498 21710 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrNqUCn/build/release/bin/kudu with pid 21789
2026-08-09T16:24:39Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1529 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786292417722232-21710-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 16:24:39 dist-test-slave-s11t krb5kdc[21941](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 16:24:39 dist-test-slave-s11t krb5kdc[21941](info): set up 2 sockets
Aug 09 16:24:39 dist-test-slave-s11t krb5kdc[21941](info): commencing operation
krb5kdc: starting...
W20260809 16:24:40.685067 21710 mini_kdc.cc:121] Time spent starting KDC: real 1.236s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786292417722232-21710-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786292417722232-21710-0/krb5kdc/test-user.keytab.
Aug 09 16:24:40 dist-test-slave-s11t krb5kdc[21941](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292680, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T16:24:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T16:24:40Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786292417722232-21710-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786292417722232-21710-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 16:24:40.809535 21710 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrNqUCn/build/release/bin/kudu
/tmp/dist-test-taskrNqUCn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786292417722232-21710-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786292417722232-21710-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786292417722232-21710-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786292417722232-21710-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46707
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786292417722232-21710-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41493
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46707
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786292417722232-21710-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786292417722232-21710-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786292417722232-21710-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:24:40.892287 21967 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:24:40.892486 21967 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:24:40.892509 21967 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:24:40.893760 21967 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 16:24:40.893821 21967 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:24:40.893837 21967 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:24:40.893868 21967 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 16:24:40.893894 21967 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 16:24:40.895390 21967 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41493
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786292417722232-21710-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786292417722232-21710-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46707
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786292417722232-21710-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46707
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786292417722232-21710-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21967
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786292417722232-21710-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:24:40.895637 21967 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:24:40.895872 21967 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:24:40.900794 21974 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:24:40.900887 21973 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:24:40.900830 21977 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:24:40.901021 21967 server_base.cc:1034] running on GCE node
I20260809 16:24:40.901258 21967 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:24:40.901571 21967 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:24:40.902729 21967 hybrid_clock.cc:648] HybridClock initialized: now 1786292680902716 us; error 49 us; skew 500 ppm
Aug 09 16:24:40 dist-test-slave-s11t krb5kdc[21941](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292680, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:24:40.905356 21967 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 16:24:40.905721 21967 webserver.cc:460] Webserver started at http://127.0.0.1:40885/ using document root <none> and password file <none>
I20260809 16:24:40.905907 21967 fs_manager.cc:359] Metadata directory not provided
I20260809 16:24:40.905949 21967 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:24:40.906038 21967 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:24:40.907006 21967 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786292417722232-21710-0/minicluster-data/master-0/data/instance:
uuid: "e1d3cd200a514c0992efc76543e00588"
format_stamp: "Formatted at 2026-08-09 16:24:40 on dist-test-slave-s11t"
server_key: "b74f9bf62565d1c6b2968ddd5e51a447"
server_key_iv: "3592693bd75ecf64d423e401039c9363"
server_key_version: "encryptionkey@0"
I20260809 16:24:40.907302 21967 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786292417722232-21710-0/minicluster-data/master-0/wal/instance:
uuid: "e1d3cd200a514c0992efc76543e00588"
format_stamp: "Formatted at 2026-08-09 16:24:40 on dist-test-slave-s11t"
server_key: "b74f9bf62565d1c6b2968ddd5e51a447"
server_key_iv: "3592693bd75ecf64d423e401039c9363"
server_key_version: "encryptionkey@0"
I20260809 16:24:40.908627 21967 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 16:24:40.909554 21989 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:24:40.909710 21967 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 16:24:40.909802 21967 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786292417722232-21710-0/minicluster-data/master-0/data,/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786292417722232-21710-0/minicluster-data/master-0/wal
uuid: "e1d3cd200a514c0992efc76543e00588"
format_stamp: "Formatted at 2026-08-09 16:24:40 on dist-test-slave-s11t"
server_key: "b74f9bf62565d1c6b2968ddd5e51a447"
server_key_iv: "3592693bd75ecf64d423e401039c9363"
server_key_version: "encryptionkey@0"
I20260809 16:24:40.909858 21967 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786292417722232-21710-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786292417722232-21710-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786292417722232-21710-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:24:40.923533 21967 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:24:40.925436 21967 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:24:40.925547 21967 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:24:40.929486 21967 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46707
I20260809 16:24:40.929538 22059 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46707 every 8 connection(s)
I20260809 16:24:40.929891 21967 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786292417722232-21710-0/minicluster-data/master-0/data/info.pb
I20260809 16:24:40.930866 22062 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:24:40.933223 22062 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e1d3cd200a514c0992efc76543e00588: Bootstrap starting.
I20260809 16:24:40.933825 22062 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e1d3cd200a514c0992efc76543e00588: Neither blocks nor log segments found. Creating new log.
I20260809 16:24:40.934106 22062 log.cc:824] T 00000000000000000000000000000000 P e1d3cd200a514c0992efc76543e00588: Log is configured to *not* fsync() on all Append() calls
I20260809 16:24:40.934374 21710 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrNqUCn/build/release/bin/kudu as pid 21967
I20260809 16:24:40.934504 21710 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786292417722232-21710-0/minicluster-data/master-0/wal/instance
I20260809 16:24:40.934700 21710 external_mini_cluster.cc:1428] Setting key 9d65b1dc0f4ffbec98bca7f7747b8e6d
I20260809 16:24:40.934900 22062 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e1d3cd200a514c0992efc76543e00588: No bootstrap required, opened a new log
I20260809 16:24:40.936641 22062 raft_consensus.cc:348] T 00000000000000000000000000000000 P e1d3cd200a514c0992efc76543e00588 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e1d3cd200a514c0992efc76543e00588" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46707 } }
I20260809 16:24:40.936867 22062 raft_consensus.cc:374] T 00000000000000000000000000000000 P e1d3cd200a514c0992efc76543e00588 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:24:40.936959 22062 raft_consensus.cc:729] T 00000000000000000000000000000000 P e1d3cd200a514c0992efc76543e00588 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e1d3cd200a514c0992efc76543e00588, State: Initialized, Role: FOLLOWER
I20260809 16:24:40.937096 22062 consensus_queue.cc:260] T 00000000000000000000000000000000 P e1d3cd200a514c0992efc76543e00588 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e1d3cd200a514c0992efc76543e00588" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46707 } }
I20260809 16:24:40.937196 22062 raft_consensus.cc:388] T 00000000000000000000000000000000 P e1d3cd200a514c0992efc76543e00588 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 16:24:40.937242 22062 raft_consensus.cc:482] T 00000000000000000000000000000000 P e1d3cd200a514c0992efc76543e00588 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 16:24:40.937289 22062 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e1d3cd200a514c0992efc76543e00588 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:24:40.938159 22062 raft_consensus.cc:504] T 00000000000000000000000000000000 P e1d3cd200a514c0992efc76543e00588 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e1d3cd200a514c0992efc76543e00588" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46707 } }
I20260809 16:24:40.938283 22062 leader_election.cc:302] T 00000000000000000000000000000000 P e1d3cd200a514c0992efc76543e00588 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e1d3cd200a514c0992efc76543e00588; no voters:
I20260809 16:24:40.938474 22062 leader_election.cc:288] T 00000000000000000000000000000000 P e1d3cd200a514c0992efc76543e00588 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 16:24:40.938496 22068 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e1d3cd200a514c0992efc76543e00588 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:24:40.938613 22068 raft_consensus.cc:686] T 00000000000000000000000000000000 P e1d3cd200a514c0992efc76543e00588 [term 1 LEADER]: Becoming Leader. State: Replica: e1d3cd200a514c0992efc76543e00588, State: Running, Role: LEADER
I20260809 16:24:40.938745 22062 sys_catalog.cc:564] T 00000000000000000000000000000000 P e1d3cd200a514c0992efc76543e00588 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 16:24:40.938761 22068 consensus_queue.cc:237] T 00000000000000000000000000000000 P e1d3cd200a514c0992efc76543e00588 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e1d3cd200a514c0992efc76543e00588" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46707 } }
I20260809 16:24:40.939213 22069 sys_catalog.cc:455] T 00000000000000000000000000000000 P e1d3cd200a514c0992efc76543e00588 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e1d3cd200a514c0992efc76543e00588. Latest consensus state: current_term: 1 leader_uuid: "e1d3cd200a514c0992efc76543e00588" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e1d3cd200a514c0992efc76543e00588" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46707 } } }
I20260809 16:24:40.939318 22069 sys_catalog.cc:458] T 00000000000000000000000000000000 P e1d3cd200a514c0992efc76543e00588 [sys.catalog]: This master's current role is: LEADER
I20260809 16:24:40.939530 22070 sys_catalog.cc:455] T 00000000000000000000000000000000 P e1d3cd200a514c0992efc76543e00588 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e1d3cd200a514c0992efc76543e00588" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e1d3cd200a514c0992efc76543e00588" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46707 } } }
I20260809 16:24:40.939939 22070 sys_catalog.cc:458] T 00000000000000000000000000000000 P e1d3cd200a514c0992efc76543e00588 [sys.catalog]: This master's current role is: LEADER
I20260809 16:24:40.940124 22081 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 16:24:40.940827 22081 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 16:24:40.942526 22081 catalog_manager.cc:1347] Generated new cluster ID: 1c60227927904b5688aea95b96e90c64
I20260809 16:24:40.942574 22081 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 09 16:24:40 dist-test-slave-s11t krb5kdc[21941](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292680, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260809 16:24:40.946796 22067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:40.935298 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.76:56717 (local address 127.0.0.1:46707)
0809 16:24:40.935483 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0809 16:24:40.935493 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 16:24:40.935529 (+ 36us) server_negotiation.cc:407] Connection header received
0809 16:24:40.936625 (+ 1096us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:24:40.936632 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:24:40.936729 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:24:40.936896 (+ 167us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:24:40.937658 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:24:40.938798 (+ 1140us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:24:40.939916 (+ 1118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:24:40.940493 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:24:40.940615 (+ 122us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:24:40.943278 (+ 2663us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:24:40.943282 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:24:40.943282 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:24:40.943294 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:24:40.945130 (+ 1836us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:24:40.945530 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:24:40.945539 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:24:40.945540 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:24:40.945646 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:24:40.945886 (+ 240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:24:40.945887 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:24:40.945888 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:24:40.946136 (+ 248us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:24:40.946314 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0809 16:24:40.946437 (+ 123us) server_negotiation.cc:299] Negotiation successful
0809 16:24:40.946654 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":133,"thread_start_us":81,"threads_started":1}
I20260809 16:24:40.947803 21710 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 16:24:40.952761 22081 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 16:24:40.953362 22081 catalog_manager.cc:1504] Loading token signing keys...
I20260809 16:24:40.960237 22081 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e1d3cd200a514c0992efc76543e00588: Generated new TSK 0
I20260809 16:24:40.960376 22067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:40.949157 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.76:42617 (local address 127.0.0.1:46707)
0809 16:24:40.949198 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0809 16:24:40.949201 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 16:24:40.952921 (+ 3720us) server_negotiation.cc:407] Connection header received
0809 16:24:40.952997 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:24:40.952998 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:24:40.953009 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:24:40.953072 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:24:40.953708 (+ 636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:24:40.954417 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:24:40.955442 (+ 1025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:24:40.955765 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:24:40.955823 (+ 58us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:24:40.957261 (+ 1438us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:24:40.957263 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:24:40.957263 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:24:40.957270 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:24:40.958846 (+ 1576us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:24:40.959280 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:24:40.959281 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:24:40.959281 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:24:40.959343 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:24:40.959619 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:24:40.959620 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:24:40.959620 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:24:40.959743 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:24:40.959862 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0809 16:24:40.960105 (+ 243us) server_negotiation.cc:299] Negotiation successful
0809 16:24:40.960174 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260809 16:24:40.960436 22081 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 16:24:40.962216 21710 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrNqUCn/build/release/bin/kudu with pid 21967
W20260809 16:24:40.983217 21939 connection.cc:535] client connection to 127.0.0.1:46707 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:46707): Connection reset by peer (error 104)
2026-08-09T16:24:40Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (1547 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786292417722232-21710-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 16:24:41 dist-test-slave-s11t krb5kdc[22107](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 16:24:41 dist-test-slave-s11t krb5kdc[22107](info): set up 2 sockets
Aug 09 16:24:41 dist-test-slave-s11t krb5kdc[22107](info): commencing operation
krb5kdc: starting...
W20260809 16:24:42.213246 21710 mini_kdc.cc:121] Time spent starting KDC: real 1.216s user 0.005s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786292417722232-21710-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786292417722232-21710-0/krb5kdc/test-user.keytab.
Aug 09 16:24:42 dist-test-slave-s11t krb5kdc[22107](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292682, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T16:24:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T16:24:42Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786292417722232-21710-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786292417722232-21710-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 16:24:42.339052 21710 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrNqUCn/build/release/bin/kudu
/tmp/dist-test-taskrNqUCn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786292417722232-21710-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786292417722232-21710-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786292417722232-21710-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786292417722232-21710-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40281
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786292417722232-21710-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40543
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40281
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786292417722232-21710-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786292417722232-21710-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786292417722232-21710-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:24:42.421151 22126 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:24:42.421379 22126 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:24:42.421423 22126 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:24:42.422550 22126 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 16:24:42.422619 22126 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:24:42.422642 22126 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:24:42.422664 22126 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 16:24:42.422729 22126 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 16:24:42.424175 22126 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40543
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786292417722232-21710-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786292417722232-21710-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40281
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786292417722232-21710-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40281
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786292417722232-21710-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22126
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786292417722232-21710-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:24:42.424453 22126 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:24:42.424708 22126 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:24:42.429668 22142 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:24:42.429651 22135 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:24:42.429837 22126 server_base.cc:1034] running on GCE node
W20260809 16:24:42.429705 22136 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:24:42.430199 22126 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:24:42.430455 22126 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:24:42.431631 22126 hybrid_clock.cc:648] HybridClock initialized: now 1786292682431608 us; error 35 us; skew 500 ppm
Aug 09 16:24:42 dist-test-slave-s11t krb5kdc[22107](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292682, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:24:42.434285 22126 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 16:24:42.434751 22126 webserver.cc:460] Webserver started at http://127.0.0.1:41121/ using document root <none> and password file <none>
I20260809 16:24:42.434952 22126 fs_manager.cc:359] Metadata directory not provided
I20260809 16:24:42.434993 22126 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:24:42.435081 22126 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:24:42.436064 22126 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786292417722232-21710-0/minicluster-data/master-0/data/instance:
uuid: "2b14357fe120445784ca9f653111e623"
format_stamp: "Formatted at 2026-08-09 16:24:42 on dist-test-slave-s11t"
server_key: "cc2f8f8289ae770d24e23edb263b75c9"
server_key_iv: "56a9fd85334b901db51562744fd84f5c"
server_key_version: "encryptionkey@0"
I20260809 16:24:42.436362 22126 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786292417722232-21710-0/minicluster-data/master-0/wal/instance:
uuid: "2b14357fe120445784ca9f653111e623"
format_stamp: "Formatted at 2026-08-09 16:24:42 on dist-test-slave-s11t"
server_key: "cc2f8f8289ae770d24e23edb263b75c9"
server_key_iv: "56a9fd85334b901db51562744fd84f5c"
server_key_version: "encryptionkey@0"
I20260809 16:24:42.437703 22126 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 16:24:42.438428 22150 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:24:42.438629 22126 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:24:42.438717 22126 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786292417722232-21710-0/minicluster-data/master-0/data,/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786292417722232-21710-0/minicluster-data/master-0/wal
uuid: "2b14357fe120445784ca9f653111e623"
format_stamp: "Formatted at 2026-08-09 16:24:42 on dist-test-slave-s11t"
server_key: "cc2f8f8289ae770d24e23edb263b75c9"
server_key_iv: "56a9fd85334b901db51562744fd84f5c"
server_key_version: "encryptionkey@0"
I20260809 16:24:42.438772 22126 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786292417722232-21710-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786292417722232-21710-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786292417722232-21710-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:24:42.453231 22126 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:24:42.454973 22126 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:24:42.455085 22126 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:24:42.458809 22126 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40281
I20260809 16:24:42.458884 22230 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40281 every 8 connection(s)
I20260809 16:24:42.459203 22126 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786292417722232-21710-0/minicluster-data/master-0/data/info.pb
I20260809 16:24:42.460089 22231 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:24:42.462360 22231 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2b14357fe120445784ca9f653111e623: Bootstrap starting.
I20260809 16:24:42.462966 22231 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2b14357fe120445784ca9f653111e623: Neither blocks nor log segments found. Creating new log.
I20260809 16:24:42.463249 22231 log.cc:824] T 00000000000000000000000000000000 P 2b14357fe120445784ca9f653111e623: Log is configured to *not* fsync() on all Append() calls
I20260809 16:24:42.464028 22231 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2b14357fe120445784ca9f653111e623: No bootstrap required, opened a new log
I20260809 16:24:42.464112 21710 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrNqUCn/build/release/bin/kudu as pid 22126
I20260809 16:24:42.464238 21710 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786292417722232-21710-0/minicluster-data/master-0/wal/instance
I20260809 16:24:42.464465 21710 external_mini_cluster.cc:1428] Setting key e605a5a8a3845d270ec814f10c115fe3
I20260809 16:24:42.466096 22231 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2b14357fe120445784ca9f653111e623 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b14357fe120445784ca9f653111e623" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40281 } }
I20260809 16:24:42.466261 22231 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2b14357fe120445784ca9f653111e623 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:24:42.466300 22231 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2b14357fe120445784ca9f653111e623 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b14357fe120445784ca9f653111e623, State: Initialized, Role: FOLLOWER
I20260809 16:24:42.466491 22231 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2b14357fe120445784ca9f653111e623 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b14357fe120445784ca9f653111e623" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40281 } }
I20260809 16:24:42.466605 22231 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2b14357fe120445784ca9f653111e623 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 16:24:42.466652 22231 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2b14357fe120445784ca9f653111e623 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 16:24:42.466702 22231 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2b14357fe120445784ca9f653111e623 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:24:42.467249 22231 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2b14357fe120445784ca9f653111e623 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b14357fe120445784ca9f653111e623" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40281 } }
I20260809 16:24:42.467372 22231 leader_election.cc:302] T 00000000000000000000000000000000 P 2b14357fe120445784ca9f653111e623 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2b14357fe120445784ca9f653111e623; no voters:
I20260809 16:24:42.467574 22231 leader_election.cc:288] T 00000000000000000000000000000000 P 2b14357fe120445784ca9f653111e623 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 16:24:42.467599 22236 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2b14357fe120445784ca9f653111e623 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:24:42.467715 22236 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2b14357fe120445784ca9f653111e623 [term 1 LEADER]: Becoming Leader. State: Replica: 2b14357fe120445784ca9f653111e623, State: Running, Role: LEADER
I20260809 16:24:42.467849 22231 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2b14357fe120445784ca9f653111e623 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 16:24:42.467859 22236 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2b14357fe120445784ca9f653111e623 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b14357fe120445784ca9f653111e623" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40281 } }
I20260809 16:24:42.468308 22237 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2b14357fe120445784ca9f653111e623 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2b14357fe120445784ca9f653111e623" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b14357fe120445784ca9f653111e623" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40281 } } }
I20260809 16:24:42.468537 22237 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2b14357fe120445784ca9f653111e623 [sys.catalog]: This master's current role is: LEADER
I20260809 16:24:42.468472 22239 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2b14357fe120445784ca9f653111e623 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2b14357fe120445784ca9f653111e623. Latest consensus state: current_term: 1 leader_uuid: "2b14357fe120445784ca9f653111e623" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b14357fe120445784ca9f653111e623" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40281 } } }
I20260809 16:24:42.468786 22239 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2b14357fe120445784ca9f653111e623 [sys.catalog]: This master's current role is: LEADER
I20260809 16:24:42.468830 22249 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 16:24:42.469404 22249 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 16:24:42.471206 22235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:42.465028 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.76:42665 (local address 127.0.0.1:40281)
0809 16:24:42.465208 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0809 16:24:42.465217 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 16:24:42.465250 (+ 33us) server_negotiation.cc:407] Connection header received
0809 16:24:42.466080 (+ 830us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:24:42.466086 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:24:42.466134 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:24:42.466244 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:24:42.466525 (+ 281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:24:42.467908 (+ 1383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:24:42.469050 (+ 1142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:24:42.469731 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:24:42.469836 (+ 105us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:24:42.470566 (+ 730us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:24:42.470569 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:24:42.470570 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 16:24:42.470755 (+ 185us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:24:42.470786 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 16:24:42.470797 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:24:42.470891 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0809 16:24:42.470981 (+ 90us) server_negotiation.cc:299] Negotiation successful
0809 16:24:42.471010 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":131,"thread_start_us":100,"threads_started":1}
I20260809 16:24:42.472182 21710 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 16:24:42.473810 22249 catalog_manager.cc:1347] Generated new cluster ID: e5aea601b0c249ae891c41cfc5c10fd3
I20260809 16:24:42.473865 22249 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 16:24:42.480712 22235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:42.477047 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.76:51539 (local address 127.0.0.1:40281)
0809 16:24:42.477100 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0809 16:24:42.477101 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 16:24:42.477204 (+ 103us) server_negotiation.cc:407] Connection header received
0809 16:24:42.477262 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:24:42.477263 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:24:42.477273 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:24:42.477325 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:24:42.477476 (+ 151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:24:42.478166 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:24:42.479224 (+ 1058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:24:42.479542 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:24:42.479600 (+ 58us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:24:42.480426 (+ 826us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:24:42.480427 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:24:42.480427 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 16:24:42.480431 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:24:42.480445 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 16:24:42.480449 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:24:42.480504 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0809 16:24:42.480601 (+ 97us) server_negotiation.cc:299] Negotiation successful
0809 16:24:42.480615 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260809 16:24:42.492218 22249 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 16:24:42.492976 22249 catalog_manager.cc:1504] Loading token signing keys...
I20260809 16:24:42.496608 22249 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2b14357fe120445784ca9f653111e623: Generated new TSK 0
I20260809 16:24:42.496804 22249 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 16:24:42.506510 21710 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrNqUCn/build/release/bin/kudu with pid 22126
W20260809 16:24:42.516425 22105 connection.cc:535] client connection to 127.0.0.1:40281 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:40281): Connection reset by peer (error 104)
2026-08-09T16:24:42Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1533 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786292417722232-21710-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 16:24:42 dist-test-slave-s11t krb5kdc[22277](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 16:24:42 dist-test-slave-s11t krb5kdc[22277](info): set up 2 sockets
Aug 09 16:24:42 dist-test-slave-s11t krb5kdc[22277](info): commencing operation
krb5kdc: starting...
W20260809 16:24:43.753834 21710 mini_kdc.cc:121] Time spent starting KDC: real 1.223s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786292417722232-21710-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786292417722232-21710-0/krb5kdc/test-user.keytab.
Aug 09 16:24:43 dist-test-slave-s11t krb5kdc[22277](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292683, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T16:24:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T16:24:43Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786292417722232-21710-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786292417722232-21710-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 16:24:43.875288 21710 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrNqUCn/build/release/bin/kudu
/tmp/dist-test-taskrNqUCn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786292417722232-21710-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786292417722232-21710-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786292417722232-21710-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786292417722232-21710-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41237
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786292417722232-21710-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34491
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41237
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786292417722232-21710-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786292417722232-21710-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786292417722232-21710-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:24:43.963279 22297 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:24:43.963475 22297 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:24:43.963497 22297 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:24:43.964543 22297 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 16:24:43.964577 22297 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:24:43.964591 22297 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:24:43.964604 22297 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 16:24:43.964617 22297 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 16:24:43.966017 22297 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34491
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786292417722232-21710-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786292417722232-21710-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41237
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786292417722232-21710-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41237
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786292417722232-21710-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22297
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786292417722232-21710-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:24:43.966202 22297 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:24:43.966367 22297 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:24:43.971318 22306 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:24:43.971329 22309 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:24:43.971395 22311 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:24:43.971717 22297 server_base.cc:1034] running on GCE node
I20260809 16:24:43.972008 22297 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:24:43.972307 22297 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:24:43.973476 22297 hybrid_clock.cc:648] HybridClock initialized: now 1786292683973446 us; error 49 us; skew 500 ppm
Aug 09 16:24:43 dist-test-slave-s11t krb5kdc[22277](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292683, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:24:43.976070 22297 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 16:24:43.976473 22297 webserver.cc:460] Webserver started at http://127.0.0.1:41833/ using document root <none> and password file <none>
I20260809 16:24:43.976751 22297 fs_manager.cc:359] Metadata directory not provided
I20260809 16:24:43.976800 22297 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:24:43.976975 22297 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:24:43.978049 22297 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786292417722232-21710-0/minicluster-data/master-0/data/instance:
uuid: "ba82fa0f9d9b4674b76831ecd115ad49"
format_stamp: "Formatted at 2026-08-09 16:24:43 on dist-test-slave-s11t"
server_key: "384aff01d7ae1bfa9fecf2d8effabdb1"
server_key_iv: "368dd57c322929a137b79df52ae14988"
server_key_version: "encryptionkey@0"
I20260809 16:24:43.978434 22297 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786292417722232-21710-0/minicluster-data/master-0/wal/instance:
uuid: "ba82fa0f9d9b4674b76831ecd115ad49"
format_stamp: "Formatted at 2026-08-09 16:24:43 on dist-test-slave-s11t"
server_key: "384aff01d7ae1bfa9fecf2d8effabdb1"
server_key_iv: "368dd57c322929a137b79df52ae14988"
server_key_version: "encryptionkey@0"
I20260809 16:24:43.979804 22297 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 16:24:43.980599 22319 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:24:43.980804 22297 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 16:24:43.980901 22297 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786292417722232-21710-0/minicluster-data/master-0/data,/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786292417722232-21710-0/minicluster-data/master-0/wal
uuid: "ba82fa0f9d9b4674b76831ecd115ad49"
format_stamp: "Formatted at 2026-08-09 16:24:43 on dist-test-slave-s11t"
server_key: "384aff01d7ae1bfa9fecf2d8effabdb1"
server_key_iv: "368dd57c322929a137b79df52ae14988"
server_key_version: "encryptionkey@0"
I20260809 16:24:43.980957 22297 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786292417722232-21710-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786292417722232-21710-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786292417722232-21710-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:24:43.984336 22297 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:24:43.986611 22297 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:24:43.986719 22297 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:24:43.990705 22297 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41237
I20260809 16:24:43.990756 22396 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41237 every 8 connection(s)
I20260809 16:24:43.991056 22297 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786292417722232-21710-0/minicluster-data/master-0/data/info.pb
I20260809 16:24:43.992036 22397 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:24:43.994447 22397 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ba82fa0f9d9b4674b76831ecd115ad49: Bootstrap starting.
I20260809 16:24:43.995048 22397 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ba82fa0f9d9b4674b76831ecd115ad49: Neither blocks nor log segments found. Creating new log.
I20260809 16:24:43.995337 22397 log.cc:824] T 00000000000000000000000000000000 P ba82fa0f9d9b4674b76831ecd115ad49: Log is configured to *not* fsync() on all Append() calls
I20260809 16:24:43.996049 22397 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ba82fa0f9d9b4674b76831ecd115ad49: No bootstrap required, opened a new log
I20260809 16:24:43.997689 22397 raft_consensus.cc:348] T 00000000000000000000000000000000 P ba82fa0f9d9b4674b76831ecd115ad49 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba82fa0f9d9b4674b76831ecd115ad49" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41237 } }
I20260809 16:24:43.997850 22397 raft_consensus.cc:374] T 00000000000000000000000000000000 P ba82fa0f9d9b4674b76831ecd115ad49 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:24:43.997901 22397 raft_consensus.cc:729] T 00000000000000000000000000000000 P ba82fa0f9d9b4674b76831ecd115ad49 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ba82fa0f9d9b4674b76831ecd115ad49, State: Initialized, Role: FOLLOWER
I20260809 16:24:43.998081 22397 consensus_queue.cc:260] T 00000000000000000000000000000000 P ba82fa0f9d9b4674b76831ecd115ad49 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba82fa0f9d9b4674b76831ecd115ad49" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41237 } }
I20260809 16:24:43.998179 22397 raft_consensus.cc:388] T 00000000000000000000000000000000 P ba82fa0f9d9b4674b76831ecd115ad49 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 16:24:43.998224 22397 raft_consensus.cc:482] T 00000000000000000000000000000000 P ba82fa0f9d9b4674b76831ecd115ad49 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 16:24:43.998274 22397 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ba82fa0f9d9b4674b76831ecd115ad49 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:24:43.998797 22397 raft_consensus.cc:504] T 00000000000000000000000000000000 P ba82fa0f9d9b4674b76831ecd115ad49 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba82fa0f9d9b4674b76831ecd115ad49" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41237 } }
I20260809 16:24:43.998917 22397 leader_election.cc:302] T 00000000000000000000000000000000 P ba82fa0f9d9b4674b76831ecd115ad49 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ba82fa0f9d9b4674b76831ecd115ad49; no voters:
I20260809 16:24:43.999094 22397 leader_election.cc:288] T 00000000000000000000000000000000 P ba82fa0f9d9b4674b76831ecd115ad49 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 16:24:43.999141 22400 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ba82fa0f9d9b4674b76831ecd115ad49 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:24:43.999320 22400 raft_consensus.cc:686] T 00000000000000000000000000000000 P ba82fa0f9d9b4674b76831ecd115ad49 [term 1 LEADER]: Becoming Leader. State: Replica: ba82fa0f9d9b4674b76831ecd115ad49, State: Running, Role: LEADER
I20260809 16:24:43.999361 22397 sys_catalog.cc:564] T 00000000000000000000000000000000 P ba82fa0f9d9b4674b76831ecd115ad49 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 16:24:43.999485 22400 consensus_queue.cc:237] T 00000000000000000000000000000000 P ba82fa0f9d9b4674b76831ecd115ad49 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba82fa0f9d9b4674b76831ecd115ad49" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41237 } }
I20260809 16:24:44.000049 21710 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrNqUCn/build/release/bin/kudu as pid 22297
I20260809 16:24:44.000075 22401 sys_catalog.cc:455] T 00000000000000000000000000000000 P ba82fa0f9d9b4674b76831ecd115ad49 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ba82fa0f9d9b4674b76831ecd115ad49" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba82fa0f9d9b4674b76831ecd115ad49" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41237 } } }
I20260809 16:24:44.000152 22401 sys_catalog.cc:458] T 00000000000000000000000000000000 P ba82fa0f9d9b4674b76831ecd115ad49 [sys.catalog]: This master's current role is: LEADER
I20260809 16:24:44.000180 21710 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786292417722232-21710-0/minicluster-data/master-0/wal/instance
I20260809 16:24:44.000406 21710 external_mini_cluster.cc:1428] Setting key 1260d52bfd8431d0b5c6d8f2c5d0979b
I20260809 16:24:44.000505 22402 sys_catalog.cc:455] T 00000000000000000000000000000000 P ba82fa0f9d9b4674b76831ecd115ad49 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ba82fa0f9d9b4674b76831ecd115ad49. Latest consensus state: current_term: 1 leader_uuid: "ba82fa0f9d9b4674b76831ecd115ad49" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba82fa0f9d9b4674b76831ecd115ad49" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41237 } } }
I20260809 16:24:44.000567 22402 sys_catalog.cc:458] T 00000000000000000000000000000000 P ba82fa0f9d9b4674b76831ecd115ad49 [sys.catalog]: This master's current role is: LEADER
W20260809 16:24:44.001358 22426 catalog_manager.cc:1558] T 00000000000000000000000000000000 P ba82fa0f9d9b4674b76831ecd115ad49: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 16:24:44.001436 22426 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 16:24:44.002561 22417 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 16:24:44.002722 22428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:44.000942 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.76:59829 (local address 127.0.0.1:41237)
0809 16:24:44.001200 (+ 258us) server_negotiation.cc:206] Beginning negotiation
0809 16:24:44.001210 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 16:24:44.001245 (+ 35us) server_negotiation.cc:407] Connection header received
0809 16:24:44.001374 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:24:44.001380 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:24:44.001412 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:24:44.001566 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:24:44.002247 (+ 681us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:24:44.002250 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:24:44.002250 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 16:24:44.002255 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:24:44.002298 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 16:24:44.002312 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:24:44.002392 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0809 16:24:44.002428 (+ 36us) server_negotiation.cc:299] Negotiation successful
0809 16:24:44.002472 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":204,"thread_start_us":82,"threads_started":1}
I20260809 16:24:44.002821 22417 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 16:24:44.003762 21710 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 16:24:44.004801 22417 catalog_manager.cc:1347] Generated new cluster ID: ac7adf29cf124755921962036d446038
I20260809 16:24:44.004866 22417 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 16:24:44.006381 22428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:44.005302 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.76:40507 (local address 127.0.0.1:41237)
0809 16:24:44.005333 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0809 16:24:44.005334 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 16:24:44.005459 (+ 125us) server_negotiation.cc:407] Connection header received
0809 16:24:44.005525 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:24:44.005526 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:24:44.005530 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:24:44.005588 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:24:44.006160 (+ 572us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:24:44.006161 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:24:44.006161 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 16:24:44.006165 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:24:44.006179 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 16:24:44.006183 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:24:44.006235 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0809 16:24:44.006305 (+ 70us) server_negotiation.cc:299] Negotiation successful
0809 16:24:44.006317 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260809 16:24:44.022454 22417 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 16:24:44.022639 22417 catalog_manager.cc:1504] Loading token signing keys...
I20260809 16:24:44.030009 22417 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ba82fa0f9d9b4674b76831ecd115ad49: Generated new TSK 0
I20260809 16:24:44.030222 22417 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 16:24:44.034780 21710 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrNqUCn/build/release/bin/kudu with pid 22297
2026-08-09T16:24:44Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1534 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786292417722232-21710-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 16:24:44 dist-test-slave-s11t krb5kdc[22442](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 16:24:44 dist-test-slave-s11t krb5kdc[22442](info): set up 2 sockets
Aug 09 16:24:44 dist-test-slave-s11t krb5kdc[22442](info): commencing operation
krb5kdc: starting...
W20260809 16:24:45.348193 21710 mini_kdc.cc:121] Time spent starting KDC: real 1.283s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786292417722232-21710-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786292417722232-21710-0/krb5kdc/test-user.keytab.
Aug 09 16:24:45 dist-test-slave-s11t krb5kdc[22442](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292685, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T16:24:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T16:24:45Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.51.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.51.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.51.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.190.keytab.
Entry for principal kudu/127.21.51.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.190.keytab.
I20260809 16:24:45.469105 21710 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrNqUCn/build/release/bin/kudu
/tmp/dist-test-taskrNqUCn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786292417722232-21710-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786292417722232-21710-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786292417722232-21710-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786292417722232-21710-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.51.190:39375
--webserver_interface=127.21.51.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.190.keytab
--principal=kudu/127.21.51.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.51.148:37081
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.51.190:39375
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786292417722232-21710-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786292417722232-21710-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786292417722232-21710-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:24:45.552511 22463 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:24:45.552738 22463 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:24:45.552762 22463 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:24:45.553910 22463 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 16:24:45.553953 22463 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:24:45.553967 22463 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:24:45.553980 22463 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 16:24:45.554031 22463 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 16:24:45.555495 22463 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.51.148:37081
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786292417722232-21710-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786292417722232-21710-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.51.190:39375
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.190.keytab
--principal=kudu/127.21.51.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.51.190:39375
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786292417722232-21710-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.51.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22463
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786292417722232-21710-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:24:45.555750 22463 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:24:45.555987 22463 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:24:45.561120 22481 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:24:45.561129 22483 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:24:45.561391 22463 server_base.cc:1034] running on GCE node
W20260809 16:24:45.561137 22478 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:24:45.561698 22463 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:24:45.562041 22463 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:24:45.563220 22463 hybrid_clock.cc:648] HybridClock initialized: now 1786292685563206 us; error 31 us; skew 500 ppm
Aug 09 16:24:45 dist-test-slave-s11t krb5kdc[22442](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292685, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.51.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:24:45.570319 22463 init.cc:377] Logged in from keytab as kudu/127.21.51.190@KRBTEST.COM (short username kudu)
I20260809 16:24:45.570783 22463 webserver.cc:460] Webserver started at http://127.21.51.190:38739/ using document root <none> and password file <none>
I20260809 16:24:45.571039 22463 fs_manager.cc:359] Metadata directory not provided
I20260809 16:24:45.571113 22463 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:24:45.571269 22463 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:24:45.572335 22463 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786292417722232-21710-0/minicluster-data/master-0/data/instance:
uuid: "58c0fc1deab24b15b745abde456cc979"
format_stamp: "Formatted at 2026-08-09 16:24:45 on dist-test-slave-s11t"
server_key: "2536176731b4e6311eaec36f229a970f"
server_key_iv: "027816b2702734352beeadcb4c757d05"
server_key_version: "encryptionkey@0"
I20260809 16:24:45.572762 22463 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786292417722232-21710-0/minicluster-data/master-0/wal/instance:
uuid: "58c0fc1deab24b15b745abde456cc979"
format_stamp: "Formatted at 2026-08-09 16:24:45 on dist-test-slave-s11t"
server_key: "2536176731b4e6311eaec36f229a970f"
server_key_iv: "027816b2702734352beeadcb4c757d05"
server_key_version: "encryptionkey@0"
I20260809 16:24:45.574227 22463 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:24:45.574998 22492 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:24:45.575191 22463 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:24:45.575300 22463 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786292417722232-21710-0/minicluster-data/master-0/data,/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786292417722232-21710-0/minicluster-data/master-0/wal
uuid: "58c0fc1deab24b15b745abde456cc979"
format_stamp: "Formatted at 2026-08-09 16:24:45 on dist-test-slave-s11t"
server_key: "2536176731b4e6311eaec36f229a970f"
server_key_iv: "027816b2702734352beeadcb4c757d05"
server_key_version: "encryptionkey@0"
I20260809 16:24:45.575376 22463 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786292417722232-21710-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786292417722232-21710-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786292417722232-21710-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:24:45.591305 22463 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:24:45.593313 22463 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:24:45.593482 22463 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:24:45.597733 22463 rpc_server.cc:307] RPC server started. Bound to: 127.21.51.190:39375
I20260809 16:24:45.597860 22574 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.51.190:39375 every 8 connection(s)
I20260809 16:24:45.598171 22463 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786292417722232-21710-0/minicluster-data/master-0/data/info.pb
I20260809 16:24:45.599100 22575 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:24:45.601405 22575 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 58c0fc1deab24b15b745abde456cc979: Bootstrap starting.
I20260809 16:24:45.601943 22575 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 58c0fc1deab24b15b745abde456cc979: Neither blocks nor log segments found. Creating new log.
I20260809 16:24:45.602180 22575 log.cc:824] T 00000000000000000000000000000000 P 58c0fc1deab24b15b745abde456cc979: Log is configured to *not* fsync() on all Append() calls
I20260809 16:24:45.602891 22575 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 58c0fc1deab24b15b745abde456cc979: No bootstrap required, opened a new log
I20260809 16:24:45.604058 21710 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrNqUCn/build/release/bin/kudu as pid 22463
I20260809 16:24:45.604151 21710 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786292417722232-21710-0/minicluster-data/master-0/wal/instance
I20260809 16:24:45.604336 21710 external_mini_cluster.cc:1428] Setting key 0f1c3d4d1b9ecc1b3484e94508b0bd25
I20260809 16:24:45.604723 22575 raft_consensus.cc:348] T 00000000000000000000000000000000 P 58c0fc1deab24b15b745abde456cc979 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "58c0fc1deab24b15b745abde456cc979" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 39375 } }
I20260809 16:24:45.605012 22575 raft_consensus.cc:374] T 00000000000000000000000000000000 P 58c0fc1deab24b15b745abde456cc979 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:24:45.605108 22575 raft_consensus.cc:729] T 00000000000000000000000000000000 P 58c0fc1deab24b15b745abde456cc979 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 58c0fc1deab24b15b745abde456cc979, State: Initialized, Role: FOLLOWER
I20260809 16:24:45.605274 22575 consensus_queue.cc:260] T 00000000000000000000000000000000 P 58c0fc1deab24b15b745abde456cc979 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "58c0fc1deab24b15b745abde456cc979" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 39375 } }
I20260809 16:24:45.605365 22575 raft_consensus.cc:388] T 00000000000000000000000000000000 P 58c0fc1deab24b15b745abde456cc979 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 16:24:45.605399 22575 raft_consensus.cc:482] T 00000000000000000000000000000000 P 58c0fc1deab24b15b745abde456cc979 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 16:24:45.605458 22575 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 58c0fc1deab24b15b745abde456cc979 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:24:45.606474 22575 raft_consensus.cc:504] T 00000000000000000000000000000000 P 58c0fc1deab24b15b745abde456cc979 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "58c0fc1deab24b15b745abde456cc979" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 39375 } }
I20260809 16:24:45.606583 22575 leader_election.cc:302] T 00000000000000000000000000000000 P 58c0fc1deab24b15b745abde456cc979 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 58c0fc1deab24b15b745abde456cc979; no voters:
I20260809 16:24:45.606741 22575 leader_election.cc:288] T 00000000000000000000000000000000 P 58c0fc1deab24b15b745abde456cc979 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 16:24:45.606801 22582 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 58c0fc1deab24b15b745abde456cc979 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:24:45.606981 22582 raft_consensus.cc:686] T 00000000000000000000000000000000 P 58c0fc1deab24b15b745abde456cc979 [term 1 LEADER]: Becoming Leader. State: Replica: 58c0fc1deab24b15b745abde456cc979, State: Running, Role: LEADER
I20260809 16:24:45.607098 22582 consensus_queue.cc:237] T 00000000000000000000000000000000 P 58c0fc1deab24b15b745abde456cc979 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "58c0fc1deab24b15b745abde456cc979" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 39375 } }
I20260809 16:24:45.606989 22575 sys_catalog.cc:564] T 00000000000000000000000000000000 P 58c0fc1deab24b15b745abde456cc979 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 16:24:45.607901 22583 sys_catalog.cc:455] T 00000000000000000000000000000000 P 58c0fc1deab24b15b745abde456cc979 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 58c0fc1deab24b15b745abde456cc979. Latest consensus state: current_term: 1 leader_uuid: "58c0fc1deab24b15b745abde456cc979" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "58c0fc1deab24b15b745abde456cc979" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 39375 } } }
I20260809 16:24:45.607929 22584 sys_catalog.cc:455] T 00000000000000000000000000000000 P 58c0fc1deab24b15b745abde456cc979 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "58c0fc1deab24b15b745abde456cc979" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "58c0fc1deab24b15b745abde456cc979" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 39375 } } }
I20260809 16:24:45.608136 22583 sys_catalog.cc:458] T 00000000000000000000000000000000 P 58c0fc1deab24b15b745abde456cc979 [sys.catalog]: This master's current role is: LEADER
I20260809 16:24:45.608167 22584 sys_catalog.cc:458] T 00000000000000000000000000000000 P 58c0fc1deab24b15b745abde456cc979 [sys.catalog]: This master's current role is: LEADER
I20260809 16:24:45.608539 22599 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 16:24:45.609113 22599 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 16:24:45.611076 22599 catalog_manager.cc:1347] Generated new cluster ID: 9e61e379d17e4083bc65764b56c7a115
I20260809 16:24:45.611145 22599 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 09 16:24:45 dist-test-slave-s11t krb5kdc[22442](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292685, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.51.190@KRBTEST.COM
I20260809 16:24:45.616057 22581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:45.604929 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43071 (local address 127.21.51.190:39375)
0809 16:24:45.605110 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0809 16:24:45.605119 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 16:24:45.605155 (+ 36us) server_negotiation.cc:407] Connection header received
0809 16:24:45.605278 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:24:45.605286 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:24:45.605341 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:24:45.605485 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:24:45.606154 (+ 669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:24:45.607556 (+ 1402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:24:45.608668 (+ 1112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:24:45.609279 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:24:45.609373 (+ 94us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:24:45.612284 (+ 2911us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:24:45.612287 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:24:45.612288 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:24:45.612298 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:24:45.614299 (+ 2001us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:24:45.614763 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:24:45.614766 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:24:45.614766 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:24:45.614882 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:24:45.615142 (+ 260us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:24:45.615143 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:24:45.615143 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:24:45.615434 (+ 291us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:24:45.615595 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0809 16:24:45.615768 (+ 173us) server_negotiation.cc:299] Negotiation successful
0809 16:24:45.615865 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":128,"thread_start_us":94,"threads_started":1}
I20260809 16:24:45.617112 21710 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 16:24:45.630475 22599 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 16:24:45.631326 22599 catalog_manager.cc:1504] Loading token signing keys...
I20260809 16:24:45.637939 22581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:45.630175 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59471 (local address 127.21.51.190:39375)
0809 16:24:45.630220 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0809 16:24:45.630222 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:24:45.630437 (+ 215us) server_negotiation.cc:407] Connection header received
0809 16:24:45.630560 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:24:45.630562 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:24:45.630575 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:24:45.630648 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:24:45.631344 (+ 696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:24:45.632109 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:24:45.633118 (+ 1009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:24:45.633469 (+ 351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:24:45.633544 (+ 75us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:24:45.634970 (+ 1426us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:24:45.634971 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:24:45.634972 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:24:45.634979 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:24:45.636647 (+ 1668us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:24:45.637087 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:24:45.637088 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:24:45.637088 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:24:45.637156 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:24:45.637385 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:24:45.637386 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:24:45.637386 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:24:45.637519 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:24:45.637639 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0809 16:24:45.637772 (+ 133us) server_negotiation.cc:299] Negotiation successful
0809 16:24:45.637844 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260809 16:24:45.640918 22599 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 58c0fc1deab24b15b745abde456cc979: Generated new TSK 0
I20260809 16:24:45.641114 22599 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 16:24:45.645921 21710 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrNqUCn/build/release/bin/kudu with pid 22463
W20260809 16:24:45.658212 22439 connection.cc:535] client connection to 127.21.51.190:39375 recv error: Network error: failed to read from TLS socket (remote: 127.21.51.190:39375): Connection reset by peer (error 104)
2026-08-09T16:24:45Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (1607 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786292417722232-21710-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 16:24:45 dist-test-slave-s11t krb5kdc[22624](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 16:24:45 dist-test-slave-s11t krb5kdc[22624](info): set up 2 sockets
Aug 09 16:24:45 dist-test-slave-s11t krb5kdc[22624](info): commencing operation
krb5kdc: starting...
W20260809 16:24:46.891603 21710 mini_kdc.cc:121] Time spent starting KDC: real 1.220s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786292417722232-21710-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786292417722232-21710-0/krb5kdc/test-user.keytab.
Aug 09 16:24:46 dist-test-slave-s11t krb5kdc[22624](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292686, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T16:24:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T16:24:46Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.51.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.51.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.51.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.190.keytab.
Entry for principal kudu/127.21.51.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.190.keytab.
I20260809 16:24:47.013906 21710 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrNqUCn/build/release/bin/kudu
/tmp/dist-test-taskrNqUCn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786292417722232-21710-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786292417722232-21710-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786292417722232-21710-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786292417722232-21710-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.51.190:44663
--webserver_interface=127.21.51.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.190.keytab
--principal=kudu/127.21.51.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.51.148:40805
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.51.190:44663
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786292417722232-21710-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786292417722232-21710-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786292417722232-21710-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:24:47.099766 22650 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:24:47.100116 22650 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:24:47.100171 22650 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:24:47.102110 22650 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 16:24:47.102175 22650 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:24:47.102201 22650 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:24:47.102232 22650 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 16:24:47.102262 22650 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 16:24:47.105338 22650 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.51.148:40805
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786292417722232-21710-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786292417722232-21710-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.51.190:44663
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.190.keytab
--principal=kudu/127.21.51.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.51.190:44663
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786292417722232-21710-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.51.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22650
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786292417722232-21710-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:24:47.105551 22650 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:24:47.105792 22650 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:24:47.110937 22660 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:24:47.110978 22663 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:24:47.110998 22661 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:24:47.111217 22650 server_base.cc:1034] running on GCE node
I20260809 16:24:47.111465 22650 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:24:47.111754 22650 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:24:47.112926 22650 hybrid_clock.cc:648] HybridClock initialized: now 1786292687112921 us; error 40 us; skew 500 ppm
Aug 09 16:24:47 dist-test-slave-s11t krb5kdc[22624](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292687, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.51.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:24:47.115541 22650 init.cc:377] Logged in from keytab as kudu/127.21.51.190@KRBTEST.COM (short username kudu)
I20260809 16:24:47.115978 22650 webserver.cc:460] Webserver started at http://127.21.51.190:34845/ using document root <none> and password file <none>
I20260809 16:24:47.116220 22650 fs_manager.cc:359] Metadata directory not provided
I20260809 16:24:47.116293 22650 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:24:47.116446 22650 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:24:47.117586 22650 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786292417722232-21710-0/minicluster-data/master-0/data/instance:
uuid: "6ac5271dc8fb4a9691d28b0443820536"
format_stamp: "Formatted at 2026-08-09 16:24:47 on dist-test-slave-s11t"
server_key: "018c3cd80b619f280136952b1e5ca3b8"
server_key_iv: "1255c1d6637e24f27f86f9826020d5d9"
server_key_version: "encryptionkey@0"
I20260809 16:24:47.117959 22650 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786292417722232-21710-0/minicluster-data/master-0/wal/instance:
uuid: "6ac5271dc8fb4a9691d28b0443820536"
format_stamp: "Formatted at 2026-08-09 16:24:47 on dist-test-slave-s11t"
server_key: "018c3cd80b619f280136952b1e5ca3b8"
server_key_iv: "1255c1d6637e24f27f86f9826020d5d9"
server_key_version: "encryptionkey@0"
I20260809 16:24:47.119310 22650 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260809 16:24:47.119987 22671 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:24:47.120165 22650 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 16:24:47.120281 22650 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786292417722232-21710-0/minicluster-data/master-0/data,/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786292417722232-21710-0/minicluster-data/master-0/wal
uuid: "6ac5271dc8fb4a9691d28b0443820536"
format_stamp: "Formatted at 2026-08-09 16:24:47 on dist-test-slave-s11t"
server_key: "018c3cd80b619f280136952b1e5ca3b8"
server_key_iv: "1255c1d6637e24f27f86f9826020d5d9"
server_key_version: "encryptionkey@0"
I20260809 16:24:47.120357 22650 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786292417722232-21710-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786292417722232-21710-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786292417722232-21710-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:24:47.133752 22650 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:24:47.136202 22650 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:24:47.136365 22650 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:24:47.141187 22650 rpc_server.cc:307] RPC server started. Bound to: 127.21.51.190:44663
I20260809 16:24:47.141222 22746 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.51.190:44663 every 8 connection(s)
I20260809 16:24:47.141593 22650 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786292417722232-21710-0/minicluster-data/master-0/data/info.pb
I20260809 16:24:47.142524 22748 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:24:47.144881 22748 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6ac5271dc8fb4a9691d28b0443820536: Bootstrap starting.
I20260809 16:24:47.145459 22748 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6ac5271dc8fb4a9691d28b0443820536: Neither blocks nor log segments found. Creating new log.
I20260809 16:24:47.145749 22748 log.cc:824] T 00000000000000000000000000000000 P 6ac5271dc8fb4a9691d28b0443820536: Log is configured to *not* fsync() on all Append() calls
I20260809 16:24:47.146488 22748 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6ac5271dc8fb4a9691d28b0443820536: No bootstrap required, opened a new log
I20260809 16:24:47.148061 22748 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6ac5271dc8fb4a9691d28b0443820536 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ac5271dc8fb4a9691d28b0443820536" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 44663 } }
I20260809 16:24:47.148203 22748 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6ac5271dc8fb4a9691d28b0443820536 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:24:47.148226 22748 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6ac5271dc8fb4a9691d28b0443820536 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6ac5271dc8fb4a9691d28b0443820536, State: Initialized, Role: FOLLOWER
I20260809 16:24:47.148432 22748 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6ac5271dc8fb4a9691d28b0443820536 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ac5271dc8fb4a9691d28b0443820536" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 44663 } }
I20260809 16:24:47.148496 22748 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6ac5271dc8fb4a9691d28b0443820536 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 16:24:47.148553 22748 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6ac5271dc8fb4a9691d28b0443820536 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 16:24:47.148604 22748 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6ac5271dc8fb4a9691d28b0443820536 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:24:47.149150 21710 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrNqUCn/build/release/bin/kudu as pid 22650
I20260809 16:24:47.149250 21710 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786292417722232-21710-0/minicluster-data/master-0/wal/instance
I20260809 16:24:47.149468 21710 external_mini_cluster.cc:1428] Setting key 2ba616f2214bb5022b1cbf0134768992
I20260809 16:24:47.149646 22748 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6ac5271dc8fb4a9691d28b0443820536 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ac5271dc8fb4a9691d28b0443820536" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 44663 } }
I20260809 16:24:47.149763 22748 leader_election.cc:302] T 00000000000000000000000000000000 P 6ac5271dc8fb4a9691d28b0443820536 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6ac5271dc8fb4a9691d28b0443820536; no voters:
I20260809 16:24:47.149956 22748 leader_election.cc:288] T 00000000000000000000000000000000 P 6ac5271dc8fb4a9691d28b0443820536 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 16:24:47.149979 22752 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6ac5271dc8fb4a9691d28b0443820536 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:24:47.150156 22752 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6ac5271dc8fb4a9691d28b0443820536 [term 1 LEADER]: Becoming Leader. State: Replica: 6ac5271dc8fb4a9691d28b0443820536, State: Running, Role: LEADER
I20260809 16:24:47.150265 22752 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6ac5271dc8fb4a9691d28b0443820536 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ac5271dc8fb4a9691d28b0443820536" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 44663 } }
I20260809 16:24:47.150699 22752 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6ac5271dc8fb4a9691d28b0443820536 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6ac5271dc8fb4a9691d28b0443820536" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ac5271dc8fb4a9691d28b0443820536" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 44663 } } }
I20260809 16:24:47.150781 22752 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6ac5271dc8fb4a9691d28b0443820536 [sys.catalog]: This master's current role is: LEADER
I20260809 16:24:47.150784 22758 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6ac5271dc8fb4a9691d28b0443820536 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6ac5271dc8fb4a9691d28b0443820536. Latest consensus state: current_term: 1 leader_uuid: "6ac5271dc8fb4a9691d28b0443820536" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ac5271dc8fb4a9691d28b0443820536" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 44663 } } }
I20260809 16:24:47.150903 22758 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6ac5271dc8fb4a9691d28b0443820536 [sys.catalog]: This master's current role is: LEADER
I20260809 16:24:47.151132 22748 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6ac5271dc8fb4a9691d28b0443820536 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 16:24:47.151249 22762 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 16:24:47.151810 22762 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 16:24:47.154712 22762 catalog_manager.cc:1347] Generated new cluster ID: b03756ce0aec4fb58805c3620080c23f
I20260809 16:24:47.154765 22762 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 16:24:47.155591 22754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:47.149981 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50321 (local address 127.21.51.190:44663)
0809 16:24:47.150147 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0809 16:24:47.150159 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 16:24:47.150198 (+ 39us) server_negotiation.cc:407] Connection header received
0809 16:24:47.150336 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:24:47.150344 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:24:47.150397 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:24:47.150563 (+ 166us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:24:47.150756 (+ 193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:24:47.152185 (+ 1429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:24:47.153435 (+ 1250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:24:47.154009 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:24:47.154125 (+ 116us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:24:47.154974 (+ 849us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:24:47.154978 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:24:47.154978 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 16:24:47.155162 (+ 184us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:24:47.155192 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 16:24:47.155203 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:24:47.155307 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0809 16:24:47.155391 (+ 84us) server_negotiation.cc:299] Negotiation successful
0809 16:24:47.155420 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":111,"thread_start_us":77,"threads_started":1}
I20260809 16:24:47.157842 21710 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 16:24:47.164067 22754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:47.160999 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33823 (local address 127.21.51.190:44663)
0809 16:24:47.161038 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0809 16:24:47.161040 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:24:47.161295 (+ 255us) server_negotiation.cc:407] Connection header received
0809 16:24:47.161336 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:24:47.161336 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:24:47.161347 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:24:47.161413 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:24:47.161571 (+ 158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:24:47.162058 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:24:47.162803 (+ 745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:24:47.163054 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:24:47.163135 (+ 81us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:24:47.163836 (+ 701us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:24:47.163838 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:24:47.163838 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 16:24:47.163843 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:24:47.163858 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 16:24:47.163863 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:24:47.163918 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0809 16:24:47.163978 (+ 60us) server_negotiation.cc:299] Negotiation successful
0809 16:24:47.163990 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260809 16:24:47.188074 22762 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 16:24:47.188674 22762 catalog_manager.cc:1504] Loading token signing keys...
I20260809 16:24:47.194957 22762 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6ac5271dc8fb4a9691d28b0443820536: Generated new TSK 0
I20260809 16:24:47.195168 22762 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 16:24:47.202790 21710 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrNqUCn/build/release/bin/kudu with pid 22650
W20260809 16:24:47.216923 22620 connection.cc:535] client connection to 127.21.51.190:44663 recv error: Network error: failed to read from TLS socket (remote: 127.21.51.190:44663): Connection reset by peer (error 104)
2026-08-09T16:24:47Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1558 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786292417722232-21710-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 16:24:47 dist-test-slave-s11t krb5kdc[22796](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 16:24:47 dist-test-slave-s11t krb5kdc[22796](info): set up 2 sockets
Aug 09 16:24:47 dist-test-slave-s11t krb5kdc[22796](info): commencing operation
krb5kdc: starting...
W20260809 16:24:48.470590 21710 mini_kdc.cc:121] Time spent starting KDC: real 1.240s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786292417722232-21710-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786292417722232-21710-0/krb5kdc/test-user.keytab.
Aug 09 16:24:48 dist-test-slave-s11t krb5kdc[22796](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292688, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T16:24:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T16:24:48Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.51.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.51.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.51.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.190.keytab.
Entry for principal kudu/127.21.51.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.190.keytab.
I20260809 16:24:48.591536 21710 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrNqUCn/build/release/bin/kudu
/tmp/dist-test-taskrNqUCn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786292417722232-21710-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786292417722232-21710-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786292417722232-21710-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786292417722232-21710-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.51.190:33103
--webserver_interface=127.21.51.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.190.keytab
--principal=kudu/127.21.51.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.51.148:45875
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.51.190:33103
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786292417722232-21710-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786292417722232-21710-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786292417722232-21710-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:24:48.672467 22819 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:24:48.672668 22819 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:24:48.672760 22819 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:24:48.673959 22819 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 16:24:48.674002 22819 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:24:48.674016 22819 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:24:48.674072 22819 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 16:24:48.674090 22819 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 16:24:48.675555 22819 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.51.148:45875
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786292417722232-21710-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786292417722232-21710-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.51.190:33103
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.190.keytab
--principal=kudu/127.21.51.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.51.190:33103
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786292417722232-21710-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.51.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22819
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786292417722232-21710-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:24:48.675803 22819 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:24:48.676028 22819 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:24:48.680918 22831 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:24:48.680867 22825 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:24:48.681098 22819 server_base.cc:1034] running on GCE node
W20260809 16:24:48.680876 22828 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:24:48.681370 22819 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:24:48.681651 22819 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:24:48.682806 22819 hybrid_clock.cc:648] HybridClock initialized: now 1786292688682798 us; error 32 us; skew 500 ppm
Aug 09 16:24:48 dist-test-slave-s11t krb5kdc[22796](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292688, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.51.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:24:48.685283 22819 init.cc:377] Logged in from keytab as kudu/127.21.51.190@KRBTEST.COM (short username kudu)
I20260809 16:24:48.685739 22819 webserver.cc:460] Webserver started at http://127.21.51.190:35305/ using document root <none> and password file <none>
I20260809 16:24:48.685971 22819 fs_manager.cc:359] Metadata directory not provided
I20260809 16:24:48.686018 22819 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:24:48.686177 22819 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:24:48.687189 22819 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786292417722232-21710-0/minicluster-data/master-0/data/instance:
uuid: "28161d2f3e16471882e066f6fc8719e6"
format_stamp: "Formatted at 2026-08-09 16:24:48 on dist-test-slave-s11t"
server_key: "e5ac9deaaa01869efe67969b53bc2386"
server_key_iv: "be56ca47260b77408ce62e6d70231501"
server_key_version: "encryptionkey@0"
I20260809 16:24:48.687546 22819 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786292417722232-21710-0/minicluster-data/master-0/wal/instance:
uuid: "28161d2f3e16471882e066f6fc8719e6"
format_stamp: "Formatted at 2026-08-09 16:24:48 on dist-test-slave-s11t"
server_key: "e5ac9deaaa01869efe67969b53bc2386"
server_key_iv: "be56ca47260b77408ce62e6d70231501"
server_key_version: "encryptionkey@0"
I20260809 16:24:48.688916 22819 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 16:24:48.689625 22844 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:24:48.689801 22819 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 16:24:48.689911 22819 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786292417722232-21710-0/minicluster-data/master-0/data,/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786292417722232-21710-0/minicluster-data/master-0/wal
uuid: "28161d2f3e16471882e066f6fc8719e6"
format_stamp: "Formatted at 2026-08-09 16:24:48 on dist-test-slave-s11t"
server_key: "e5ac9deaaa01869efe67969b53bc2386"
server_key_iv: "be56ca47260b77408ce62e6d70231501"
server_key_version: "encryptionkey@0"
I20260809 16:24:48.689985 22819 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786292417722232-21710-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786292417722232-21710-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786292417722232-21710-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:24:48.693231 22819 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:24:48.695019 22819 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:24:48.695165 22819 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:24:48.699194 22819 rpc_server.cc:307] RPC server started. Bound to: 127.21.51.190:33103
I20260809 16:24:48.699229 22932 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.51.190:33103 every 8 connection(s)
I20260809 16:24:48.699564 22819 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786292417722232-21710-0/minicluster-data/master-0/data/info.pb
I20260809 16:24:48.700557 22933 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:24:48.702775 22933 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 28161d2f3e16471882e066f6fc8719e6: Bootstrap starting.
I20260809 16:24:48.703392 22933 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 28161d2f3e16471882e066f6fc8719e6: Neither blocks nor log segments found. Creating new log.
I20260809 16:24:48.703624 22933 log.cc:824] T 00000000000000000000000000000000 P 28161d2f3e16471882e066f6fc8719e6: Log is configured to *not* fsync() on all Append() calls
I20260809 16:24:48.704288 22933 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 28161d2f3e16471882e066f6fc8719e6: No bootstrap required, opened a new log
I20260809 16:24:48.705964 21710 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrNqUCn/build/release/bin/kudu as pid 22819
I20260809 16:24:48.706087 21710 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786292417722232-21710-0/minicluster-data/master-0/wal/instance
I20260809 16:24:48.706043 22933 raft_consensus.cc:348] T 00000000000000000000000000000000 P 28161d2f3e16471882e066f6fc8719e6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28161d2f3e16471882e066f6fc8719e6" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 33103 } }
I20260809 16:24:48.706212 22933 raft_consensus.cc:374] T 00000000000000000000000000000000 P 28161d2f3e16471882e066f6fc8719e6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:24:48.706262 22933 raft_consensus.cc:729] T 00000000000000000000000000000000 P 28161d2f3e16471882e066f6fc8719e6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 28161d2f3e16471882e066f6fc8719e6, State: Initialized, Role: FOLLOWER
I20260809 16:24:48.706319 21710 external_mini_cluster.cc:1428] Setting key cf86b7c0802bacb4d44dbcb1799609ac
I20260809 16:24:48.706405 22933 consensus_queue.cc:260] T 00000000000000000000000000000000 P 28161d2f3e16471882e066f6fc8719e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28161d2f3e16471882e066f6fc8719e6" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 33103 } }
I20260809 16:24:48.706497 22933 raft_consensus.cc:388] T 00000000000000000000000000000000 P 28161d2f3e16471882e066f6fc8719e6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 16:24:48.706534 22933 raft_consensus.cc:482] T 00000000000000000000000000000000 P 28161d2f3e16471882e066f6fc8719e6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 16:24:48.706589 22933 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 28161d2f3e16471882e066f6fc8719e6 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:24:48.707664 22933 raft_consensus.cc:504] T 00000000000000000000000000000000 P 28161d2f3e16471882e066f6fc8719e6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28161d2f3e16471882e066f6fc8719e6" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 33103 } }
I20260809 16:24:48.707805 22933 leader_election.cc:302] T 00000000000000000000000000000000 P 28161d2f3e16471882e066f6fc8719e6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 28161d2f3e16471882e066f6fc8719e6; no voters:
I20260809 16:24:48.707973 22933 leader_election.cc:288] T 00000000000000000000000000000000 P 28161d2f3e16471882e066f6fc8719e6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 16:24:48.708001 22938 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 28161d2f3e16471882e066f6fc8719e6 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:24:48.708135 22938 raft_consensus.cc:686] T 00000000000000000000000000000000 P 28161d2f3e16471882e066f6fc8719e6 [term 1 LEADER]: Becoming Leader. State: Replica: 28161d2f3e16471882e066f6fc8719e6, State: Running, Role: LEADER
I20260809 16:24:48.708257 22933 sys_catalog.cc:564] T 00000000000000000000000000000000 P 28161d2f3e16471882e066f6fc8719e6 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 16:24:48.708294 22938 consensus_queue.cc:237] T 00000000000000000000000000000000 P 28161d2f3e16471882e066f6fc8719e6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28161d2f3e16471882e066f6fc8719e6" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 33103 } }
I20260809 16:24:48.708530 22937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:48.706833 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50315 (local address 127.21.51.190:33103)
0809 16:24:48.707033 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0809 16:24:48.707042 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 16:24:48.707077 (+ 35us) server_negotiation.cc:407] Connection header received
0809 16:24:48.707172 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:24:48.707177 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:24:48.707198 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:24:48.707302 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:24:48.707948 (+ 646us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:24:48.707952 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:24:48.707952 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 16:24:48.707957 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:24:48.708006 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 16:24:48.708022 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:24:48.708091 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0809 16:24:48.708318 (+ 227us) server_negotiation.cc:299] Negotiation successful
0809 16:24:48.708358 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":149,"thread_start_us":119,"threads_started":1}
I20260809 16:24:48.708650 22938 sys_catalog.cc:455] T 00000000000000000000000000000000 P 28161d2f3e16471882e066f6fc8719e6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 28161d2f3e16471882e066f6fc8719e6. Latest consensus state: current_term: 1 leader_uuid: "28161d2f3e16471882e066f6fc8719e6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28161d2f3e16471882e066f6fc8719e6" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 33103 } } }
I20260809 16:24:48.708756 22939 sys_catalog.cc:455] T 00000000000000000000000000000000 P 28161d2f3e16471882e066f6fc8719e6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "28161d2f3e16471882e066f6fc8719e6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28161d2f3e16471882e066f6fc8719e6" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 33103 } } }
I20260809 16:24:48.708922 22939 sys_catalog.cc:458] T 00000000000000000000000000000000 P 28161d2f3e16471882e066f6fc8719e6 [sys.catalog]: This master's current role is: LEADER
I20260809 16:24:48.708822 22938 sys_catalog.cc:458] T 00000000000000000000000000000000 P 28161d2f3e16471882e066f6fc8719e6 [sys.catalog]: This master's current role is: LEADER
I20260809 16:24:48.709154 22953 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 16:24:48.709424 21710 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 16:24:48.709741 22953 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 16:24:48.712041 22937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:48.711000 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55557 (local address 127.21.51.190:33103)
0809 16:24:48.711039 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0809 16:24:48.711041 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:24:48.711155 (+ 114us) server_negotiation.cc:407] Connection header received
0809 16:24:48.711216 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:24:48.711217 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:24:48.711221 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:24:48.711278 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:24:48.711841 (+ 563us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:24:48.711842 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:24:48.711843 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 16:24:48.711847 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:24:48.711862 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 16:24:48.711866 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:24:48.711934 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0809 16:24:48.711970 (+ 36us) server_negotiation.cc:299] Negotiation successful
0809 16:24:48.711983 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260809 16:24:48.713739 22953 catalog_manager.cc:1347] Generated new cluster ID: cde8d6edea984e51a4ef76ca09e14a5a
I20260809 16:24:48.713783 22953 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 16:24:48.723506 22953 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 16:24:48.723675 22953 catalog_manager.cc:1504] Loading token signing keys...
I20260809 16:24:48.728641 22953 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 28161d2f3e16471882e066f6fc8719e6: Generated new TSK 0
I20260809 16:24:48.728819 22953 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 16:24:48.731747 21710 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrNqUCn/build/release/bin/kudu with pid 22819
2026-08-09T16:24:48Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1525 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786292417722232-21710-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 16:24:48 dist-test-slave-s11t krb5kdc[22984](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 16:24:48 dist-test-slave-s11t krb5kdc[22984](info): set up 2 sockets
Aug 09 16:24:48 dist-test-slave-s11t krb5kdc[22984](info): commencing operation
krb5kdc: starting...
W20260809 16:24:49.979580 21710 mini_kdc.cc:121] Time spent starting KDC: real 1.223s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786292417722232-21710-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786292417722232-21710-0/krb5kdc/test-user.keytab.
Aug 09 16:24:50 dist-test-slave-s11t krb5kdc[22984](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292690, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T16:24:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T16:24:50Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.51.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.51.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.51.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.190.keytab.
Entry for principal kudu/127.21.51.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.190.keytab.
I20260809 16:24:50.105180 21710 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrNqUCn/build/release/bin/kudu
/tmp/dist-test-taskrNqUCn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786292417722232-21710-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786292417722232-21710-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786292417722232-21710-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786292417722232-21710-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.51.190:41585
--webserver_interface=127.21.51.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.190.keytab
--principal=kudu/127.21.51.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.51.148:37243
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.51.190:41585
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786292417722232-21710-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786292417722232-21710-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786292417722232-21710-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:24:50.188329 23001 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:24:50.188576 23001 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:24:50.188617 23001 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:24:50.189881 23001 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 16:24:50.189955 23001 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:24:50.189982 23001 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:24:50.190016 23001 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 16:24:50.190038 23001 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 16:24:50.191524 23001 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.51.148:37243
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786292417722232-21710-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786292417722232-21710-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.51.190:41585
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.190.keytab
--principal=kudu/127.21.51.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.51.190:41585
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786292417722232-21710-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.51.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23001
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786292417722232-21710-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:24:50.191802 23001 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:24:50.192035 23001 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:24:50.196982 23008 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:24:50.197077 23001 server_base.cc:1034] running on GCE node
W20260809 16:24:50.197052 23009 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:24:50.197007 23014 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:24:50.197484 23001 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:24:50.197803 23001 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:24:50.198954 23001 hybrid_clock.cc:648] HybridClock initialized: now 1786292690198937 us; error 44 us; skew 500 ppm
Aug 09 16:24:50 dist-test-slave-s11t krb5kdc[22984](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292690, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.51.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:24:50.205214 23001 init.cc:377] Logged in from keytab as kudu/127.21.51.190@KRBTEST.COM (short username kudu)
I20260809 16:24:50.205619 23001 webserver.cc:460] Webserver started at http://127.21.51.190:41029/ using document root <none> and password file <none>
I20260809 16:24:50.205803 23001 fs_manager.cc:359] Metadata directory not provided
I20260809 16:24:50.205843 23001 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:24:50.205935 23001 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:24:50.206959 23001 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786292417722232-21710-0/minicluster-data/master-0/data/instance:
uuid: "e94c72899b5e45849a28fd46304c45e6"
format_stamp: "Formatted at 2026-08-09 16:24:50 on dist-test-slave-s11t"
server_key: "f4295ca36c6156ab67218e16c0c4ef66"
server_key_iv: "61e85d9418f48dcfb621846903969f3f"
server_key_version: "encryptionkey@0"
I20260809 16:24:50.207267 23001 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786292417722232-21710-0/minicluster-data/master-0/wal/instance:
uuid: "e94c72899b5e45849a28fd46304c45e6"
format_stamp: "Formatted at 2026-08-09 16:24:50 on dist-test-slave-s11t"
server_key: "f4295ca36c6156ab67218e16c0c4ef66"
server_key_iv: "61e85d9418f48dcfb621846903969f3f"
server_key_version: "encryptionkey@0"
I20260809 16:24:50.208552 23001 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 16:24:50.209331 23023 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:24:50.209506 23001 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 16:24:50.209619 23001 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786292417722232-21710-0/minicluster-data/master-0/data,/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786292417722232-21710-0/minicluster-data/master-0/wal
uuid: "e94c72899b5e45849a28fd46304c45e6"
format_stamp: "Formatted at 2026-08-09 16:24:50 on dist-test-slave-s11t"
server_key: "f4295ca36c6156ab67218e16c0c4ef66"
server_key_iv: "61e85d9418f48dcfb621846903969f3f"
server_key_version: "encryptionkey@0"
I20260809 16:24:50.209733 23001 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786292417722232-21710-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786292417722232-21710-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786292417722232-21710-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:24:50.222663 23001 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:24:50.224485 23001 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:24:50.224593 23001 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:24:50.228586 23001 rpc_server.cc:307] RPC server started. Bound to: 127.21.51.190:41585
I20260809 16:24:50.228638 23104 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.51.190:41585 every 8 connection(s)
I20260809 16:24:50.229018 23001 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786292417722232-21710-0/minicluster-data/master-0/data/info.pb
I20260809 16:24:50.229912 23105 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:24:50.230023 21710 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrNqUCn/build/release/bin/kudu as pid 23001
I20260809 16:24:50.230152 21710 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786292417722232-21710-0/minicluster-data/master-0/wal/instance
I20260809 16:24:50.230345 21710 external_mini_cluster.cc:1428] Setting key de037689464b7c814d0ba43ceaeec54c
I20260809 16:24:50.233443 23105 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e94c72899b5e45849a28fd46304c45e6: Bootstrap starting.
I20260809 16:24:50.234131 23105 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e94c72899b5e45849a28fd46304c45e6: Neither blocks nor log segments found. Creating new log.
I20260809 16:24:50.234427 23105 log.cc:824] T 00000000000000000000000000000000 P e94c72899b5e45849a28fd46304c45e6: Log is configured to *not* fsync() on all Append() calls
I20260809 16:24:50.235183 23105 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e94c72899b5e45849a28fd46304c45e6: No bootstrap required, opened a new log
Aug 09 16:24:50 dist-test-slave-s11t krb5kdc[22984](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292690, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.51.190@KRBTEST.COM
I20260809 16:24:50.237247 23105 raft_consensus.cc:348] T 00000000000000000000000000000000 P e94c72899b5e45849a28fd46304c45e6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e94c72899b5e45849a28fd46304c45e6" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 41585 } }
I20260809 16:24:50.237442 23105 raft_consensus.cc:374] T 00000000000000000000000000000000 P e94c72899b5e45849a28fd46304c45e6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:24:50.237520 23105 raft_consensus.cc:729] T 00000000000000000000000000000000 P e94c72899b5e45849a28fd46304c45e6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e94c72899b5e45849a28fd46304c45e6, State: Initialized, Role: FOLLOWER
I20260809 16:24:50.237673 23105 consensus_queue.cc:260] T 00000000000000000000000000000000 P e94c72899b5e45849a28fd46304c45e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e94c72899b5e45849a28fd46304c45e6" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 41585 } }
I20260809 16:24:50.237772 23105 raft_consensus.cc:388] T 00000000000000000000000000000000 P e94c72899b5e45849a28fd46304c45e6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 16:24:50.237833 23105 raft_consensus.cc:482] T 00000000000000000000000000000000 P e94c72899b5e45849a28fd46304c45e6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 16:24:50.237887 23105 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e94c72899b5e45849a28fd46304c45e6 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:24:50.238587 23105 raft_consensus.cc:504] T 00000000000000000000000000000000 P e94c72899b5e45849a28fd46304c45e6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e94c72899b5e45849a28fd46304c45e6" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 41585 } }
I20260809 16:24:50.238721 23105 leader_election.cc:302] T 00000000000000000000000000000000 P e94c72899b5e45849a28fd46304c45e6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e94c72899b5e45849a28fd46304c45e6; no voters:
I20260809 16:24:50.238932 23105 leader_election.cc:288] T 00000000000000000000000000000000 P e94c72899b5e45849a28fd46304c45e6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 16:24:50.238991 23116 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e94c72899b5e45849a28fd46304c45e6 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:24:50.239183 23116 raft_consensus.cc:686] T 00000000000000000000000000000000 P e94c72899b5e45849a28fd46304c45e6 [term 1 LEADER]: Becoming Leader. State: Replica: e94c72899b5e45849a28fd46304c45e6, State: Running, Role: LEADER
I20260809 16:24:50.239267 23105 sys_catalog.cc:564] T 00000000000000000000000000000000 P e94c72899b5e45849a28fd46304c45e6 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 16:24:50.239328 23116 consensus_queue.cc:237] T 00000000000000000000000000000000 P e94c72899b5e45849a28fd46304c45e6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e94c72899b5e45849a28fd46304c45e6" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 41585 } }
I20260809 16:24:50.239761 23116 sys_catalog.cc:455] T 00000000000000000000000000000000 P e94c72899b5e45849a28fd46304c45e6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e94c72899b5e45849a28fd46304c45e6. Latest consensus state: current_term: 1 leader_uuid: "e94c72899b5e45849a28fd46304c45e6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e94c72899b5e45849a28fd46304c45e6" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 41585 } } }
I20260809 16:24:50.240010 23116 sys_catalog.cc:458] T 00000000000000000000000000000000 P e94c72899b5e45849a28fd46304c45e6 [sys.catalog]: This master's current role is: LEADER
I20260809 16:24:50.239920 23118 sys_catalog.cc:455] T 00000000000000000000000000000000 P e94c72899b5e45849a28fd46304c45e6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e94c72899b5e45849a28fd46304c45e6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e94c72899b5e45849a28fd46304c45e6" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 41585 } } }
I20260809 16:24:50.240454 23118 sys_catalog.cc:458] T 00000000000000000000000000000000 P e94c72899b5e45849a28fd46304c45e6 [sys.catalog]: This master's current role is: LEADER
W20260809 16:24:50.241158 23136 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e94c72899b5e45849a28fd46304c45e6: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 16:24:50.241219 23136 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 16:24:50.241325 23128 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 16:24:50.241592 23128 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 16:24:50.241935 23110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:50.230884 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36821 (local address 127.21.51.190:41585)
0809 16:24:50.231069 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0809 16:24:50.231078 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 16:24:50.231110 (+ 32us) server_negotiation.cc:407] Connection header received
0809 16:24:50.231624 (+ 514us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:24:50.231634 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:24:50.231698 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:24:50.231851 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:24:50.232457 (+ 606us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:24:50.233903 (+ 1446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:24:50.234732 (+ 829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:24:50.235248 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:24:50.235370 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:24:50.237858 (+ 2488us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:24:50.237864 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:24:50.237869 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:24:50.237898 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:24:50.239915 (+ 2017us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:24:50.240373 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:24:50.240376 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:24:50.240377 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:24:50.240489 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:24:50.240868 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:24:50.240869 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:24:50.240869 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:24:50.241130 (+ 261us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:24:50.241276 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0809 16:24:50.241572 (+ 296us) server_negotiation.cc:299] Negotiation successful
0809 16:24:50.241715 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":136,"thread_start_us":80,"threads_started":1}
I20260809 16:24:50.243340 21710 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 16:24:50.243513 23128 catalog_manager.cc:1347] Generated new cluster ID: ec504cfa9daa4cbdb814e09887311702
I20260809 16:24:50.243563 23128 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 16:24:50.256569 23128 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 16:24:50.257434 23128 catalog_manager.cc:1504] Loading token signing keys...
I20260809 16:24:50.261816 23128 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e94c72899b5e45849a28fd46304c45e6: Generated new TSK 0
I20260809 16:24:50.262066 23128 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 16:24:50.265689 23110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:50.245060 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50095 (local address 127.21.51.190:41585)
0809 16:24:50.245116 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0809 16:24:50.245119 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 16:24:50.257508 (+ 12389us) server_negotiation.cc:407] Connection header received
0809 16:24:50.257639 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:24:50.257640 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:24:50.257652 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:24:50.257687 (+ 35us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:24:50.258491 (+ 804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:24:50.259298 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:24:50.260420 (+ 1122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:24:50.260808 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:24:50.260850 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:24:50.262650 (+ 1800us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:24:50.262652 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:24:50.262653 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:24:50.262660 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:24:50.264318 (+ 1658us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:24:50.264799 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:24:50.264800 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:24:50.264801 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:24:50.264871 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:24:50.265105 (+ 234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:24:50.265106 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:24:50.265106 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:24:50.265229 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:24:50.265340 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0809 16:24:50.265497 (+ 157us) server_negotiation.cc:299] Negotiation successful
0809 16:24:50.265584 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260809 16:24:50.267668 21710 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrNqUCn/build/release/bin/kudu with pid 23001
2026-08-09T16:24:50Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (1537 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786292417722232-21710-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 16:24:50 dist-test-slave-s11t krb5kdc[23151](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 16:24:50 dist-test-slave-s11t krb5kdc[23151](info): set up 2 sockets
Aug 09 16:24:50 dist-test-slave-s11t krb5kdc[23151](info): commencing operation
krb5kdc: starting...
W20260809 16:24:51.517760 21710 mini_kdc.cc:121] Time spent starting KDC: real 1.224s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786292417722232-21710-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786292417722232-21710-0/krb5kdc/test-user.keytab.
Aug 09 16:24:51 dist-test-slave-s11t krb5kdc[23151](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292691, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T16:24:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T16:24:51Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.51.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.51.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.51.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.190.keytab.
Entry for principal kudu/127.21.51.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.190.keytab.
I20260809 16:24:51.647588 21710 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrNqUCn/build/release/bin/kudu
/tmp/dist-test-taskrNqUCn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786292417722232-21710-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786292417722232-21710-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786292417722232-21710-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786292417722232-21710-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.51.190:34771
--webserver_interface=127.21.51.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.190.keytab
--principal=kudu/127.21.51.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.51.148:42059
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.51.190:34771
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786292417722232-21710-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786292417722232-21710-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786292417722232-21710-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:24:51.729271 23167 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:24:51.729511 23167 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:24:51.729548 23167 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:24:51.730717 23167 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 16:24:51.730790 23167 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:24:51.730818 23167 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:24:51.730868 23167 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 16:24:51.730898 23167 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 16:24:51.732431 23167 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.51.148:42059
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786292417722232-21710-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786292417722232-21710-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.51.190:34771
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.190.keytab
--principal=kudu/127.21.51.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.51.190:34771
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786292417722232-21710-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.51.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23167
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786292417722232-21710-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:24:51.732741 23167 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:24:51.732988 23167 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:24:51.737819 23176 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:24:51.737948 23177 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:24:51.737884 23181 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:24:51.738350 23167 server_base.cc:1034] running on GCE node
I20260809 16:24:51.738591 23167 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:24:51.738863 23167 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:24:51.740033 23167 hybrid_clock.cc:648] HybridClock initialized: now 1786292691740024 us; error 40 us; skew 500 ppm
Aug 09 16:24:51 dist-test-slave-s11t krb5kdc[23151](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292691, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.51.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:24:51.746990 23167 init.cc:377] Logged in from keytab as kudu/127.21.51.190@KRBTEST.COM (short username kudu)
I20260809 16:24:51.747440 23167 webserver.cc:460] Webserver started at http://127.21.51.190:33347/ using document root <none> and password file <none>
I20260809 16:24:51.747665 23167 fs_manager.cc:359] Metadata directory not provided
I20260809 16:24:51.747731 23167 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:24:51.747881 23167 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:24:51.749114 23167 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786292417722232-21710-0/minicluster-data/master-0/data/instance:
uuid: "b9e495f58e8a42ee9b9845d243fd2099"
format_stamp: "Formatted at 2026-08-09 16:24:51 on dist-test-slave-s11t"
server_key: "d285ac843c92c84cc7227e68de6a744d"
server_key_iv: "e33d33c352ed0d6b510e26c902ad5312"
server_key_version: "encryptionkey@0"
I20260809 16:24:51.749492 23167 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786292417722232-21710-0/minicluster-data/master-0/wal/instance:
uuid: "b9e495f58e8a42ee9b9845d243fd2099"
format_stamp: "Formatted at 2026-08-09 16:24:51 on dist-test-slave-s11t"
server_key: "d285ac843c92c84cc7227e68de6a744d"
server_key_iv: "e33d33c352ed0d6b510e26c902ad5312"
server_key_version: "encryptionkey@0"
I20260809 16:24:51.750844 23167 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 16:24:51.751560 23192 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:24:51.751736 23167 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 16:24:51.751837 23167 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786292417722232-21710-0/minicluster-data/master-0/data,/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786292417722232-21710-0/minicluster-data/master-0/wal
uuid: "b9e495f58e8a42ee9b9845d243fd2099"
format_stamp: "Formatted at 2026-08-09 16:24:51 on dist-test-slave-s11t"
server_key: "d285ac843c92c84cc7227e68de6a744d"
server_key_iv: "e33d33c352ed0d6b510e26c902ad5312"
server_key_version: "encryptionkey@0"
I20260809 16:24:51.751914 23167 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786292417722232-21710-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786292417722232-21710-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786292417722232-21710-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:24:51.761729 23167 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:24:51.764092 23167 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:24:51.764212 23167 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:24:51.768167 23167 rpc_server.cc:307] RPC server started. Bound to: 127.21.51.190:34771
I20260809 16:24:51.768249 23266 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.51.190:34771 every 8 connection(s)
I20260809 16:24:51.768746 23167 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786292417722232-21710-0/minicluster-data/master-0/data/info.pb
I20260809 16:24:51.769687 23267 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:24:51.771963 23267 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b9e495f58e8a42ee9b9845d243fd2099: Bootstrap starting.
I20260809 16:24:51.772439 21710 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrNqUCn/build/release/bin/kudu as pid 23167
I20260809 16:24:51.772576 21710 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786292417722232-21710-0/minicluster-data/master-0/wal/instance
I20260809 16:24:51.772637 23267 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b9e495f58e8a42ee9b9845d243fd2099: Neither blocks nor log segments found. Creating new log.
I20260809 16:24:51.772821 21710 external_mini_cluster.cc:1428] Setting key f8af86ae16b8e266ed085442f4405e67
I20260809 16:24:51.772995 23267 log.cc:824] T 00000000000000000000000000000000 P b9e495f58e8a42ee9b9845d243fd2099: Log is configured to *not* fsync() on all Append() calls
I20260809 16:24:51.773806 23267 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b9e495f58e8a42ee9b9845d243fd2099: No bootstrap required, opened a new log
I20260809 16:24:51.776036 23267 raft_consensus.cc:348] T 00000000000000000000000000000000 P b9e495f58e8a42ee9b9845d243fd2099 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b9e495f58e8a42ee9b9845d243fd2099" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 34771 } }
I20260809 16:24:51.776208 23267 raft_consensus.cc:374] T 00000000000000000000000000000000 P b9e495f58e8a42ee9b9845d243fd2099 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:24:51.776239 23267 raft_consensus.cc:729] T 00000000000000000000000000000000 P b9e495f58e8a42ee9b9845d243fd2099 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b9e495f58e8a42ee9b9845d243fd2099, State: Initialized, Role: FOLLOWER
I20260809 16:24:51.776361 23267 consensus_queue.cc:260] T 00000000000000000000000000000000 P b9e495f58e8a42ee9b9845d243fd2099 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b9e495f58e8a42ee9b9845d243fd2099" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 34771 } }
I20260809 16:24:51.776434 23267 raft_consensus.cc:388] T 00000000000000000000000000000000 P b9e495f58e8a42ee9b9845d243fd2099 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 16:24:51.776463 23267 raft_consensus.cc:482] T 00000000000000000000000000000000 P b9e495f58e8a42ee9b9845d243fd2099 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 16:24:51.776510 23267 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b9e495f58e8a42ee9b9845d243fd2099 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:24:51.777159 23267 raft_consensus.cc:504] T 00000000000000000000000000000000 P b9e495f58e8a42ee9b9845d243fd2099 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b9e495f58e8a42ee9b9845d243fd2099" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 34771 } }
I20260809 16:24:51.777275 23267 leader_election.cc:302] T 00000000000000000000000000000000 P b9e495f58e8a42ee9b9845d243fd2099 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b9e495f58e8a42ee9b9845d243fd2099; no voters:
I20260809 16:24:51.777469 23267 leader_election.cc:288] T 00000000000000000000000000000000 P b9e495f58e8a42ee9b9845d243fd2099 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 16:24:51.777649 23282 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b9e495f58e8a42ee9b9845d243fd2099 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:24:51.777787 23267 sys_catalog.cc:564] T 00000000000000000000000000000000 P b9e495f58e8a42ee9b9845d243fd2099 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 16:24:51.777802 23282 raft_consensus.cc:686] T 00000000000000000000000000000000 P b9e495f58e8a42ee9b9845d243fd2099 [term 1 LEADER]: Becoming Leader. State: Replica: b9e495f58e8a42ee9b9845d243fd2099, State: Running, Role: LEADER
I20260809 16:24:51.777961 23282 consensus_queue.cc:237] T 00000000000000000000000000000000 P b9e495f58e8a42ee9b9845d243fd2099 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b9e495f58e8a42ee9b9845d243fd2099" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 34771 } }
I20260809 16:24:51.778267 23281 sys_catalog.cc:455] T 00000000000000000000000000000000 P b9e495f58e8a42ee9b9845d243fd2099 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b9e495f58e8a42ee9b9845d243fd2099" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b9e495f58e8a42ee9b9845d243fd2099" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 34771 } } }
I20260809 16:24:51.778350 23283 sys_catalog.cc:455] T 00000000000000000000000000000000 P b9e495f58e8a42ee9b9845d243fd2099 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b9e495f58e8a42ee9b9845d243fd2099. Latest consensus state: current_term: 1 leader_uuid: "b9e495f58e8a42ee9b9845d243fd2099" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b9e495f58e8a42ee9b9845d243fd2099" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 34771 } } }
I20260809 16:24:51.778421 23281 sys_catalog.cc:458] T 00000000000000000000000000000000 P b9e495f58e8a42ee9b9845d243fd2099 [sys.catalog]: This master's current role is: LEADER
I20260809 16:24:51.778520 23283 sys_catalog.cc:458] T 00000000000000000000000000000000 P b9e495f58e8a42ee9b9845d243fd2099 [sys.catalog]: This master's current role is: LEADER
I20260809 16:24:51.778855 23276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:51.773355 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38315 (local address 127.21.51.190:34771)
0809 16:24:51.773542 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0809 16:24:51.773552 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 16:24:51.773611 (+ 59us) server_negotiation.cc:407] Connection header received
0809 16:24:51.774137 (+ 526us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:24:51.774144 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:24:51.774191 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:24:51.774303 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:24:51.774552 (+ 249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:24:51.775439 (+ 887us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:24:51.776583 (+ 1144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:24:51.777086 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:24:51.777174 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:24:51.777953 (+ 779us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:24:51.777957 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:24:51.777957 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 16:24:51.778152 (+ 195us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:24:51.778192 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 16:24:51.778205 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:24:51.778544 (+ 339us) server_negotiation.cc:1035] Waiting for connection context
0809 16:24:51.778577 (+ 33us) server_negotiation.cc:299] Negotiation successful
0809 16:24:51.778650 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":130,"thread_start_us":82,"threads_started":1}
I20260809 16:24:51.779862 21710 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
W20260809 16:24:51.780026 23307 catalog_manager.cc:1558] T 00000000000000000000000000000000 P b9e495f58e8a42ee9b9845d243fd2099: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 16:24:51.780086 23307 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 16:24:51.780134 23295 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 16:24:51.780300 23295 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 16:24:51.782131 23295 catalog_manager.cc:1347] Generated new cluster ID: c8157db85c1040e58b5ad611f09dccde
I20260809 16:24:51.782188 23295 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 16:24:51.785001 23276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:51.781564 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37691 (local address 127.21.51.190:34771)
0809 16:24:51.781615 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0809 16:24:51.781617 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:24:51.781647 (+ 30us) server_negotiation.cc:407] Connection header received
0809 16:24:51.781715 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:24:51.781715 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:24:51.781723 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:24:51.781794 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:24:51.781944 (+ 150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:24:51.782635 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:24:51.783639 (+ 1004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:24:51.783927 (+ 288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:24:51.783976 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:24:51.784669 (+ 693us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:24:51.784670 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:24:51.784670 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 16:24:51.784674 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:24:51.784726 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 16:24:51.784730 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:24:51.784771 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0809 16:24:51.784916 (+ 145us) server_negotiation.cc:299] Negotiation successful
0809 16:24:51.784933 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260809 16:24:51.787779 23295 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 16:24:51.788295 23295 catalog_manager.cc:1504] Loading token signing keys...
I20260809 16:24:51.797907 23295 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b9e495f58e8a42ee9b9845d243fd2099: Generated new TSK 0
I20260809 16:24:51.798087 23295 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 16:24:51.805011 21710 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrNqUCn/build/release/bin/kudu with pid 23167
2026-08-09T16:24:51Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1537 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786292417722232-21710-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 16:24:51 dist-test-slave-s11t krb5kdc[23326](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 16:24:51 dist-test-slave-s11t krb5kdc[23326](info): set up 2 sockets
Aug 09 16:24:51 dist-test-slave-s11t krb5kdc[23326](info): commencing operation
krb5kdc: starting...
W20260809 16:24:53.090497 21710 mini_kdc.cc:121] Time spent starting KDC: real 1.259s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786292417722232-21710-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786292417722232-21710-0/krb5kdc/test-user.keytab.
Aug 09 16:24:53 dist-test-slave-s11t krb5kdc[23326](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292693, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T16:24:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T16:24:53Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.51.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.51.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.51.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.190.keytab.
Entry for principal kudu/127.21.51.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.190.keytab.
I20260809 16:24:53.223034 21710 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrNqUCn/build/release/bin/kudu
/tmp/dist-test-taskrNqUCn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786292417722232-21710-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786292417722232-21710-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786292417722232-21710-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786292417722232-21710-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.51.190:36139
--webserver_interface=127.21.51.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.190.keytab
--principal=kudu/127.21.51.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.51.148:34383
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.51.190:36139
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786292417722232-21710-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786292417722232-21710-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786292417722232-21710-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:24:53.309267 23344 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:24:53.309528 23344 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:24:53.309572 23344 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:24:53.310832 23344 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 16:24:53.310913 23344 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:24:53.310938 23344 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:24:53.310971 23344 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 16:24:53.311021 23344 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 16:24:53.312649 23344 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.51.148:34383
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786292417722232-21710-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786292417722232-21710-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.51.190:36139
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.190.keytab
--principal=kudu/127.21.51.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.51.190:36139
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786292417722232-21710-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.51.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23344
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786292417722232-21710-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:24:53.313046 23344 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:24:53.313318 23344 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:24:53.318678 23353 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:24:53.318702 23351 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:24:53.318678 23350 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:24:53.318923 23344 server_base.cc:1034] running on GCE node
I20260809 16:24:53.319180 23344 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:24:53.319552 23344 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:24:53.320768 23344 hybrid_clock.cc:648] HybridClock initialized: now 1786292693320757 us; error 55 us; skew 500 ppm
Aug 09 16:24:53 dist-test-slave-s11t krb5kdc[23326](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292693, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.51.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:24:53.323484 23344 init.cc:377] Logged in from keytab as kudu/127.21.51.190@KRBTEST.COM (short username kudu)
I20260809 16:24:53.323945 23344 webserver.cc:460] Webserver started at http://127.21.51.190:36319/ using document root <none> and password file <none>
I20260809 16:24:53.324193 23344 fs_manager.cc:359] Metadata directory not provided
I20260809 16:24:53.324266 23344 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:24:53.324426 23344 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:24:53.325577 23344 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786292417722232-21710-0/minicluster-data/master-0/data/instance:
uuid: "a874dc888e864323b782568bb7e808da"
format_stamp: "Formatted at 2026-08-09 16:24:53 on dist-test-slave-s11t"
server_key: "23df85c480ad1e726433b5ed086cccd7"
server_key_iv: "7f96d823b73369c5c268c4fce685f426"
server_key_version: "encryptionkey@0"
I20260809 16:24:53.325939 23344 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786292417722232-21710-0/minicluster-data/master-0/wal/instance:
uuid: "a874dc888e864323b782568bb7e808da"
format_stamp: "Formatted at 2026-08-09 16:24:53 on dist-test-slave-s11t"
server_key: "23df85c480ad1e726433b5ed086cccd7"
server_key_iv: "7f96d823b73369c5c268c4fce685f426"
server_key_version: "encryptionkey@0"
I20260809 16:24:53.327323 23344 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260809 16:24:53.328066 23363 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:24:53.328255 23344 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 16:24:53.328372 23344 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786292417722232-21710-0/minicluster-data/master-0/data,/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786292417722232-21710-0/minicluster-data/master-0/wal
uuid: "a874dc888e864323b782568bb7e808da"
format_stamp: "Formatted at 2026-08-09 16:24:53 on dist-test-slave-s11t"
server_key: "23df85c480ad1e726433b5ed086cccd7"
server_key_iv: "7f96d823b73369c5c268c4fce685f426"
server_key_version: "encryptionkey@0"
I20260809 16:24:53.328447 23344 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786292417722232-21710-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786292417722232-21710-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786292417722232-21710-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:24:53.331871 23344 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:24:53.334355 23344 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:24:53.334522 23344 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:24:53.338701 23344 rpc_server.cc:307] RPC server started. Bound to: 127.21.51.190:36139
I20260809 16:24:53.338769 23454 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.51.190:36139 every 8 connection(s)
I20260809 16:24:53.339113 23344 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786292417722232-21710-0/minicluster-data/master-0/data/info.pb
I20260809 16:24:53.340106 23456 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:24:53.342448 23456 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a874dc888e864323b782568bb7e808da: Bootstrap starting.
I20260809 16:24:53.343052 23456 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a874dc888e864323b782568bb7e808da: Neither blocks nor log segments found. Creating new log.
I20260809 16:24:53.343344 23456 log.cc:824] T 00000000000000000000000000000000 P a874dc888e864323b782568bb7e808da: Log is configured to *not* fsync() on all Append() calls
I20260809 16:24:53.344064 23456 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a874dc888e864323b782568bb7e808da: No bootstrap required, opened a new log
I20260809 16:24:53.345702 23456 raft_consensus.cc:348] T 00000000000000000000000000000000 P a874dc888e864323b782568bb7e808da [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a874dc888e864323b782568bb7e808da" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 36139 } }
I20260809 16:24:53.345841 23456 raft_consensus.cc:374] T 00000000000000000000000000000000 P a874dc888e864323b782568bb7e808da [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:24:53.345896 23456 raft_consensus.cc:729] T 00000000000000000000000000000000 P a874dc888e864323b782568bb7e808da [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a874dc888e864323b782568bb7e808da, State: Initialized, Role: FOLLOWER
I20260809 16:24:53.346057 23456 consensus_queue.cc:260] T 00000000000000000000000000000000 P a874dc888e864323b782568bb7e808da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a874dc888e864323b782568bb7e808da" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 36139 } }
I20260809 16:24:53.346122 23456 raft_consensus.cc:388] T 00000000000000000000000000000000 P a874dc888e864323b782568bb7e808da [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 16:24:53.346163 23456 raft_consensus.cc:482] T 00000000000000000000000000000000 P a874dc888e864323b782568bb7e808da [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 16:24:53.346216 23456 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a874dc888e864323b782568bb7e808da [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:24:53.347048 23456 raft_consensus.cc:504] T 00000000000000000000000000000000 P a874dc888e864323b782568bb7e808da [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a874dc888e864323b782568bb7e808da" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 36139 } }
I20260809 16:24:53.347163 23456 leader_election.cc:302] T 00000000000000000000000000000000 P a874dc888e864323b782568bb7e808da [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a874dc888e864323b782568bb7e808da; no voters:
I20260809 16:24:53.347345 23456 leader_election.cc:288] T 00000000000000000000000000000000 P a874dc888e864323b782568bb7e808da [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 16:24:53.347407 23459 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a874dc888e864323b782568bb7e808da [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:24:53.347581 23459 raft_consensus.cc:686] T 00000000000000000000000000000000 P a874dc888e864323b782568bb7e808da [term 1 LEADER]: Becoming Leader. State: Replica: a874dc888e864323b782568bb7e808da, State: Running, Role: LEADER
I20260809 16:24:53.347602 23456 sys_catalog.cc:564] T 00000000000000000000000000000000 P a874dc888e864323b782568bb7e808da [sys.catalog]: configured and running, proceeding with master startup.
I20260809 16:24:53.347764 23459 consensus_queue.cc:237] T 00000000000000000000000000000000 P a874dc888e864323b782568bb7e808da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a874dc888e864323b782568bb7e808da" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 36139 } }
I20260809 16:24:53.348174 23461 sys_catalog.cc:455] T 00000000000000000000000000000000 P a874dc888e864323b782568bb7e808da [sys.catalog]: SysCatalogTable state changed. Reason: New leader a874dc888e864323b782568bb7e808da. Latest consensus state: current_term: 1 leader_uuid: "a874dc888e864323b782568bb7e808da" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a874dc888e864323b782568bb7e808da" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 36139 } } }
I20260809 16:24:53.348312 23461 sys_catalog.cc:458] T 00000000000000000000000000000000 P a874dc888e864323b782568bb7e808da [sys.catalog]: This master's current role is: LEADER
I20260809 16:24:53.348171 23460 sys_catalog.cc:455] T 00000000000000000000000000000000 P a874dc888e864323b782568bb7e808da [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a874dc888e864323b782568bb7e808da" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a874dc888e864323b782568bb7e808da" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 36139 } } }
I20260809 16:24:53.348452 23460 sys_catalog.cc:458] T 00000000000000000000000000000000 P a874dc888e864323b782568bb7e808da [sys.catalog]: This master's current role is: LEADER
I20260809 16:24:53.348735 21710 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrNqUCn/build/release/bin/kudu as pid 23344
I20260809 16:24:53.348826 23471 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 16:24:53.348853 21710 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786292417722232-21710-0/minicluster-data/master-0/wal/instance
I20260809 16:24:53.349076 21710 external_mini_cluster.cc:1428] Setting key 09f5afeeaa8734584e199fc72246e6fd
I20260809 16:24:53.349363 23471 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 16:24:53.350984 23471 catalog_manager.cc:1347] Generated new cluster ID: 1a744bdc1f6b44e285c47cfe50450eb7
I20260809 16:24:53.351043 23471 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 16:24:53.352716 23487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:53.349526 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55249 (local address 127.21.51.190:36139)
0809 16:24:53.349714 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0809 16:24:53.349723 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 16:24:53.351136 (+ 1413us) server_negotiation.cc:407] Connection header received
0809 16:24:53.351366 (+ 230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:24:53.351373 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:24:53.351404 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:24:53.351566 (+ 162us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:24:53.352228 (+ 662us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:24:53.352232 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:24:53.352232 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 16:24:53.352236 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:24:53.352278 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 16:24:53.352299 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:24:53.352359 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0809 16:24:53.352467 (+ 108us) server_negotiation.cc:299] Negotiation successful
0809 16:24:53.352513 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":132,"thread_start_us":79,"threads_started":1}
I20260809 16:24:53.353627 21710 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 16:24:53.356150 23487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:53.355111 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56499 (local address 127.21.51.190:36139)
0809 16:24:53.355140 (+ 29us) server_negotiation.cc:206] Beginning negotiation
0809 16:24:53.355142 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:24:53.355245 (+ 103us) server_negotiation.cc:407] Connection header received
0809 16:24:53.355339 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:24:53.355339 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:24:53.355343 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:24:53.355369 (+ 26us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:24:53.355951 (+ 582us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:24:53.355952 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:24:53.355953 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 16:24:53.355956 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:24:53.355970 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 16:24:53.355974 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:24:53.355999 (+ 25us) server_negotiation.cc:1035] Waiting for connection context
0809 16:24:53.356085 (+ 86us) server_negotiation.cc:299] Negotiation successful
0809 16:24:53.356096 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14}
I20260809 16:24:53.359298 23471 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 16:24:53.359488 23471 catalog_manager.cc:1504] Loading token signing keys...
I20260809 16:24:53.375826 23471 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a874dc888e864323b782568bb7e808da: Generated new TSK 0
I20260809 16:24:53.376022 23471 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 16:24:53.378352 21710 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrNqUCn/build/release/bin/kudu with pid 23344
2026-08-09T16:24:53Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1576 ms)
[----------] 18 tests from AuthTokenIssuingTest (27984 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786292417722232-21710-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 16:24:53 dist-test-slave-s11t krb5kdc[23507](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 16:24:53 dist-test-slave-s11t krb5kdc[23507](info): set up 2 sockets
Aug 09 16:24:53 dist-test-slave-s11t krb5kdc[23507](info): commencing operation
krb5kdc: starting...
W20260809 16:24:54.618582 21710 mini_kdc.cc:121] Time spent starting KDC: real 1.210s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786292417722232-21710-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786292417722232-21710-0/krb5kdc/test-user.keytab.
Aug 09 16:24:54 dist-test-slave-s11t krb5kdc[23507](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292694, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T16:24:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T16:24:54Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.51.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.51.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.51.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.190.keytab.
Entry for principal kudu/127.21.51.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.190.keytab.
I20260809 16:24:54.739691 21710 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrNqUCn/build/release/bin/kudu
/tmp/dist-test-taskrNqUCn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786292417722232-21710-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786292417722232-21710-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786292417722232-21710-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786292417722232-21710-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.51.190:45837
--webserver_interface=127.21.51.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.190.keytab
--principal=kudu/127.21.51.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.51.148:44317
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.51.190:45837,127.21.51.189:44933,127.21.51.188:40535
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786292417722232-21710-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786292417722232-21710-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786292417722232-21710-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:24:54.822256 23527 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260809 16:24:54.822468 23527 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:24:54.822486 23527 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:24:54.822506 23527 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:24:54.823679 23527 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 16:24:54.823725 23527 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:24:54.823746 23527 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:24:54.823804 23527 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 16:24:54.823825 23527 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 16:24:54.825368 23527 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.51.148:44317
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786292417722232-21710-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786292417722232-21710-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.51.190:45837,127.21.51.189:44933,127.21.51.188:40535
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.190.keytab
--principal=kudu/127.21.51.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.51.190:45837
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786292417722232-21710-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.51.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23527
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786292417722232-21710-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:24:54.825636 23527 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:24:54.825884 23527 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:24:54.831002 23533 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:24:54.831002 23542 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:24:54.831005 23537 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:24:54.831130 23527 server_base.cc:1034] running on GCE node
I20260809 16:24:54.831477 23527 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:24:54.831789 23527 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:24:54.832931 23527 hybrid_clock.cc:648] HybridClock initialized: now 1786292694832910 us; error 38 us; skew 500 ppm
Aug 09 16:24:54 dist-test-slave-s11t krb5kdc[23507](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292694, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.51.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:24:54.835459 23527 init.cc:377] Logged in from keytab as kudu/127.21.51.190@KRBTEST.COM (short username kudu)
I20260809 16:24:54.835832 23527 webserver.cc:460] Webserver started at http://127.21.51.190:35689/ using document root <none> and password file <none>
I20260809 16:24:54.836038 23527 fs_manager.cc:359] Metadata directory not provided
I20260809 16:24:54.836081 23527 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:24:54.836171 23527 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:24:54.837239 23527 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786292417722232-21710-0/minicluster-data/master-0/data/instance:
uuid: "9a8d69c13e9c4d919b2fe59e4cefe39a"
format_stamp: "Formatted at 2026-08-09 16:24:54 on dist-test-slave-s11t"
server_key: "808bc7e235d371028f7f203a877b8684"
server_key_iv: "d07e239b8845540f76206dc220f11b73"
server_key_version: "encryptionkey@0"
I20260809 16:24:54.837550 23527 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786292417722232-21710-0/minicluster-data/master-0/wal/instance:
uuid: "9a8d69c13e9c4d919b2fe59e4cefe39a"
format_stamp: "Formatted at 2026-08-09 16:24:54 on dist-test-slave-s11t"
server_key: "808bc7e235d371028f7f203a877b8684"
server_key_iv: "d07e239b8845540f76206dc220f11b73"
server_key_version: "encryptionkey@0"
I20260809 16:24:54.838835 23527 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 16:24:54.839607 23551 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:24:54.839960 23527 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:24:54.840050 23527 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786292417722232-21710-0/minicluster-data/master-0/data,/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786292417722232-21710-0/minicluster-data/master-0/wal
uuid: "9a8d69c13e9c4d919b2fe59e4cefe39a"
format_stamp: "Formatted at 2026-08-09 16:24:54 on dist-test-slave-s11t"
server_key: "808bc7e235d371028f7f203a877b8684"
server_key_iv: "d07e239b8845540f76206dc220f11b73"
server_key_version: "encryptionkey@0"
I20260809 16:24:54.840166 23527 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786292417722232-21710-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786292417722232-21710-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786292417722232-21710-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:24:54.848475 23527 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:24:54.850245 23527 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:24:54.850409 23527 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:24:54.854487 23527 rpc_server.cc:307] RPC server started. Bound to: 127.21.51.190:45837
I20260809 16:24:54.854557 23639 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.51.190:45837 every 8 connection(s)
I20260809 16:24:54.854887 23527 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786292417722232-21710-0/minicluster-data/master-0/data/info.pb
I20260809 16:24:54.855868 23641 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:24:54.857481 23641 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.51.190" port: 45837 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 16:24:54 dist-test-slave-s11t krb5kdc[23507](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292694, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.51.190@KRBTEST.COM for kudu/127.21.51.190@KRBTEST.COM
I20260809 16:24:54.864880 21710 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrNqUCn/build/release/bin/kudu as pid 23527
I20260809 16:24:54.865008 21710 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786292417722232-21710-0/minicluster-data/master-0/wal/instance
I20260809 16:24:54.865198 21710 external_mini_cluster.cc:1428] Setting key aaa1edc81ff95b28a5550a10ad51acae
I20260809 16:24:54.866870 23649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:54.858210 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48468 (local address 127.21.51.190:45837)
0809 16:24:54.858364 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0809 16:24:54.858368 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 16:24:54.858457 (+ 89us) server_negotiation.cc:407] Connection header received
0809 16:24:54.858964 (+ 507us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:24:54.858966 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:24:54.858982 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:24:54.859066 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:24:54.859568 (+ 502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:24:54.860383 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:24:54.861272 (+ 889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:24:54.861583 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:24:54.861663 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:24:54.863624 (+ 1961us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:24:54.863632 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:24:54.863632 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:24:54.863638 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:24:54.865398 (+ 1760us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:24:54.865821 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:24:54.865823 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:24:54.865823 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:24:54.865902 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:24:54.866152 (+ 250us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:24:54.866152 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:24:54.866152 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:24:54.866342 (+ 190us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:24:54.866415 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0809 16:24:54.866626 (+ 211us) server_negotiation.cc:299] Negotiation successful
0809 16:24:54.866718 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":119,"thread_start_us":83,"threads_started":1}
I20260809 16:24:54.866994 23644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:54.858143 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:45837 (local address 127.0.0.1:48468)
0809 16:24:54.858314 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0809 16:24:54.858356 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0809 16:24:54.858804 (+ 448us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:24:54.859092 (+ 288us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:24:54.859095 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:24:54.859332 (+ 237us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:24:54.859484 (+ 152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:24:54.859488 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:24:54.860470 (+ 982us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:24:54.860471 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:24:54.861200 (+ 729us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:24:54.861202 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:24:54.861678 (+ 476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:24:54.861679 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:24:54.861732 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:24:54.862186 (+ 454us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:24:54.862194 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:24:54.863545 (+ 1351us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:24:54.865475 (+ 1930us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:24:54.865482 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:24:54.865484 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:24:54.865744 (+ 260us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:24:54.865970 (+ 226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:24:54.865971 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:24:54.865972 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:24:54.866086 (+ 114us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:24:54.866410 (+ 324us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:24:54.866416 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:24:54.866490 (+ 74us) client_negotiation.cc:769] Sending connection context
0809 16:24:54.866589 (+ 99us) client_negotiation.cc:240] Negotiation successful
0809 16:24:54.866800 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"mutex_wait_us":54,"thread_start_us":95,"threads_started":1}
I20260809 16:24:54.867897 23641 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.51.189" port: 44933 } has no permanent_uuid. Determining permanent_uuid...
I20260809 16:24:54.868234 23644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:54.868106 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.189:44933 (local address 127.0.0.1:44736)
0809 16:24:54.868155 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0809 16:24:54.868205 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.51.189:44933: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260809 16:24:54.868391 23552 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.21.51.189:44933: connect: Connection refused (error 111)
I20260809 16:24:54.868863 23644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:54.868759 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.189:44933 (local address 127.0.0.1:44740)
0809 16:24:54.868798 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0809 16:24:54.868828 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.51.189:44933: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":18}
W20260809 16:24:54.869035 23641 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.51.189:44933: Network error: Client connection negotiation failed: client connection to 127.21.51.189:44933: connect: Connection refused (error 111)
WARNING: no policy specified for kudu/127.21.51.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.51.189@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.51.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.189.keytab.
Entry for principal kudu/127.21.51.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.189.keytab.
I20260809 16:24:54.889178 21710 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrNqUCn/build/release/bin/kudu
/tmp/dist-test-taskrNqUCn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786292417722232-21710-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786292417722232-21710-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786292417722232-21710-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786292417722232-21710-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.51.189:44933
--webserver_interface=127.21.51.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.189.keytab
--principal=kudu/127.21.51.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.51.148:44317
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.51.190:45837,127.21.51.189:44933,127.21.51.188:40535
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786292417722232-21710-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786292417722232-21710-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786292417722232-21710-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 16:24:54.918274 23641 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.51.189" port: 44933 } attempt: 1
I20260809 16:24:54.918769 23644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:54.918629 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.189:44933 (local address 127.0.0.1:44746)
0809 16:24:54.918696 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0809 16:24:54.918736 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.51.189:44933: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260809 16:24:54.919144 23644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:54.919038 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.189:44933 (local address 127.0.0.1:44754)
0809 16:24:54.919090 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0809 16:24:54.919122 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.51.189:44933: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260809 16:24:54.919241 23641 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.51.189:44933: Network error: Client connection negotiation failed: client connection to 127.21.51.189:44933: connect: Connection refused (error 111)
W20260809 16:24:54.971619 23657 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260809 16:24:54.971887 23657 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:24:54.971920 23657 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:24:54.971957 23657 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:24:54.973122 23657 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 16:24:54.973188 23657 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:24:54.973213 23657 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:24:54.973234 23657 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 16:24:54.973263 23657 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 16:24:54.974735 23657 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.51.148:44317
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786292417722232-21710-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786292417722232-21710-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.21.51.190:45837,127.21.51.189:44933,127.21.51.188:40535
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.189.keytab
--principal=kudu/127.21.51.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.51.189:44933
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786292417722232-21710-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.21.51.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23657
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786292417722232-21710-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:24:54.975026 23657 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:24:54.975257 23657 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:24:54.980238 23666 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:24:54.980402 23669 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:24:54.980247 23667 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:24:54.980490 23657 server_base.cc:1034] running on GCE node
I20260809 16:24:54.980795 23657 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:24:54.981148 23657 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:24:54.982311 23657 hybrid_clock.cc:648] HybridClock initialized: now 1786292694982301 us; error 41 us; skew 500 ppm
Aug 09 16:24:54 dist-test-slave-s11t krb5kdc[23507](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292694, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.51.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:24:54.984860 23657 init.cc:377] Logged in from keytab as kudu/127.21.51.189@KRBTEST.COM (short username kudu)
I20260809 16:24:54.985347 23657 webserver.cc:460] Webserver started at http://127.21.51.189:41471/ using document root <none> and password file <none>
I20260809 16:24:54.985577 23657 fs_manager.cc:359] Metadata directory not provided
I20260809 16:24:54.985646 23657 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:24:54.985783 23657 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:24:54.986829 23657 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786292417722232-21710-0/minicluster-data/master-1/data/instance:
uuid: "4fc0017b5efd45aebd25c4c6a6ac994b"
format_stamp: "Formatted at 2026-08-09 16:24:54 on dist-test-slave-s11t"
server_key: "11649d45b44c287a4ace43edff9f3985"
server_key_iv: "1120c7c440ea8380e46d392ff8edaf4d"
server_key_version: "encryptionkey@0"
I20260809 16:24:54.987207 23657 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786292417722232-21710-0/minicluster-data/master-1/wal/instance:
uuid: "4fc0017b5efd45aebd25c4c6a6ac994b"
format_stamp: "Formatted at 2026-08-09 16:24:54 on dist-test-slave-s11t"
server_key: "11649d45b44c287a4ace43edff9f3985"
server_key_iv: "1120c7c440ea8380e46d392ff8edaf4d"
server_key_version: "encryptionkey@0"
I20260809 16:24:54.987390 23641 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.51.189" port: 44933 } attempt: 2
I20260809 16:24:54.987799 23644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:54.987641 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.189:44933 (local address 127.0.0.1:44762)
0809 16:24:54.987713 (+ 72us) negotiation.cc:107] Waiting for socket to connect
0809 16:24:54.987754 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.51.189:44933: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260809 16:24:54.988188 23644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:54.988083 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.189:44933 (local address 127.0.0.1:44778)
0809 16:24:54.988134 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0809 16:24:54.988163 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.51.189:44933: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260809 16:24:54.988323 23641 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.51.189:44933: Network error: Client connection negotiation failed: client connection to 127.21.51.189:44933: connect: Connection refused (error 111)
I20260809 16:24:54.988989 23657 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.003s sys 0.001s
I20260809 16:24:54.989714 23678 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:24:54.989928 23657 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 16:24:54.990020 23657 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786292417722232-21710-0/minicluster-data/master-1/data,/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786292417722232-21710-0/minicluster-data/master-1/wal
uuid: "4fc0017b5efd45aebd25c4c6a6ac994b"
format_stamp: "Formatted at 2026-08-09 16:24:54 on dist-test-slave-s11t"
server_key: "11649d45b44c287a4ace43edff9f3985"
server_key_iv: "1120c7c440ea8380e46d392ff8edaf4d"
server_key_version: "encryptionkey@0"
I20260809 16:24:54.990095 23657 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786292417722232-21710-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786292417722232-21710-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786292417722232-21710-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:24:55.009354 23657 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:24:55.011334 23657 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:24:55.011493 23657 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:24:55.015844 23657 rpc_server.cc:307] RPC server started. Bound to: 127.21.51.189:44933
I20260809 16:24:55.015942 23749 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.51.189:44933 every 8 connection(s)
I20260809 16:24:55.016276 23657 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786292417722232-21710-0/minicluster-data/master-1/data/info.pb
I20260809 16:24:55.017385 23750 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:24:55.018894 23750 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.51.190" port: 45837 } has no permanent_uuid. Determining permanent_uuid...
I20260809 16:24:55.024475 21710 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrNqUCn/build/release/bin/kudu as pid 23657
I20260809 16:24:55.024596 21710 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786292417722232-21710-0/minicluster-data/master-1/wal/instance
I20260809 16:24:55.024828 21710 external_mini_cluster.cc:1428] Setting key 3b4eb76f9e66025060e469c7d5b513af
Aug 09 16:24:55 dist-test-slave-s11t krb5kdc[23507](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292694, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.51.189@KRBTEST.COM for kudu/127.21.51.190@KRBTEST.COM
I20260809 16:24:55.028753 23649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:55.019573 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48484 (local address 127.21.51.190:45837)
0809 16:24:55.019641 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0809 16:24:55.019643 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:24:55.019836 (+ 193us) server_negotiation.cc:407] Connection header received
0809 16:24:55.020275 (+ 439us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:24:55.020276 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:24:55.020285 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:24:55.020345 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:24:55.021251 (+ 906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:24:55.021739 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:24:55.022718 (+ 979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:24:55.022906 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:24:55.022958 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:24:55.025545 (+ 2587us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:24:55.025547 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:24:55.025547 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:24:55.025554 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:24:55.027348 (+ 1794us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:24:55.027730 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:24:55.027732 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:24:55.027732 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:24:55.027806 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:24:55.028029 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:24:55.028030 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:24:55.028030 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:24:55.028182 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:24:55.028264 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0809 16:24:55.028497 (+ 233us) server_negotiation.cc:299] Negotiation successful
0809 16:24:55.028594 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260809 16:24:55.028780 23753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:55.019526 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:45837 (local address 127.0.0.1:48484)
0809 16:24:55.019702 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0809 16:24:55.019745 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0809 16:24:55.020194 (+ 449us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:24:55.020434 (+ 240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:24:55.020437 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:24:55.020757 (+ 320us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:24:55.021160 (+ 403us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:24:55.021166 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:24:55.021831 (+ 665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:24:55.021832 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:24:55.022632 (+ 800us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:24:55.022634 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:24:55.022990 (+ 356us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:24:55.022991 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:24:55.023097 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:24:55.023634 (+ 537us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:24:55.023642 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:24:55.025429 (+ 1787us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:24:55.027430 (+ 2001us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:24:55.027436 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:24:55.027438 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:24:55.027652 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:24:55.027875 (+ 223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:24:55.027876 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:24:55.027876 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:24:55.027977 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:24:55.028250 (+ 273us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:24:55.028255 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:24:55.028364 (+ 109us) client_negotiation.cc:769] Sending connection context
0809 16:24:55.028451 (+ 87us) client_negotiation.cc:240] Negotiation successful
0809 16:24:55.028567 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":102,"threads_started":1}
I20260809 16:24:55.029352 23750 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.51.189" port: 44933 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 16:24:55 dist-test-slave-s11t krb5kdc[23507](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292694, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.51.189@KRBTEST.COM for kudu/127.21.51.189@KRBTEST.COM
WARNING: no policy specified for kudu/127.21.51.188@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.51.188@KRBTEST.COM" created.
I20260809 16:24:55.036953 23755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:55.029702 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44780 (local address 127.21.51.189:44933)
0809 16:24:55.029865 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0809 16:24:55.029868 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 16:24:55.029887 (+ 19us) server_negotiation.cc:407] Connection header received
0809 16:24:55.029904 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:24:55.029905 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:24:55.029916 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:24:55.030001 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:24:55.030269 (+ 268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:24:55.030923 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:24:55.031596 (+ 673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:24:55.031891 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:24:55.031969 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:24:55.034076 (+ 2107us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:24:55.034089 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:24:55.034089 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:24:55.034095 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:24:55.035579 (+ 1484us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:24:55.035988 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:24:55.035993 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:24:55.035993 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:24:55.036070 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:24:55.036272 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:24:55.036273 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:24:55.036273 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:24:55.036422 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:24:55.036534 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0809 16:24:55.036729 (+ 195us) server_negotiation.cc:299] Negotiation successful
0809 16:24:55.036852 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":48,"server-negotiator.queue_time_us":135,"thread_start_us":86,"threads_started":1}
I20260809 16:24:55.036996 23753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:55.029606 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.189:44933 (local address 127.0.0.1:44780)
0809 16:24:55.029656 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0809 16:24:55.029666 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 16:24:55.029703 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:24:55.030001 (+ 298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:24:55.030002 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:24:55.030139 (+ 137us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:24:55.030185 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:24:55.030186 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:24:55.031013 (+ 827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:24:55.031014 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:24:55.031504 (+ 490us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:24:55.031505 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:24:55.031965 (+ 460us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:24:55.031966 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:24:55.032030 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:24:55.032486 (+ 456us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:24:55.032490 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:24:55.033956 (+ 1466us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:24:55.035674 (+ 1718us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:24:55.035675 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:24:55.035675 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:24:55.035893 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:24:55.036143 (+ 250us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:24:55.036144 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:24:55.036144 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:24:55.036206 (+ 62us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:24:55.036530 (+ 324us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:24:55.036531 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:24:55.036579 (+ 48us) client_negotiation.cc:769] Sending connection context
0809 16:24:55.036660 (+ 81us) client_negotiation.cc:240] Negotiation successful
0809 16:24:55.036793 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33}
I20260809 16:24:55.037686 23750 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.51.188" port: 40535 } has no permanent_uuid. Determining permanent_uuid...
I20260809 16:24:55.038053 23753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:55.037920 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.188:40535 (local address 127.0.0.1:42874)
0809 16:24:55.037970 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0809 16:24:55.038021 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.51.188:40535: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260809 16:24:55.038209 23680 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.21.51.188:40535: connect: Connection refused (error 111)
I20260809 16:24:55.038674 23753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:55.038576 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.188:40535 (local address 127.0.0.1:42884)
0809 16:24:55.038614 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0809 16:24:55.038646 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.51.188:40535: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":20}
W20260809 16:24:55.038795 23750 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.51.188:40535: Network error: Client connection negotiation failed: client connection to 127.21.51.188:40535: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.51.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.188.keytab.
Entry for principal kudu/127.21.51.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.188.keytab.
I20260809 16:24:55.051406 21710 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrNqUCn/build/release/bin/kudu
/tmp/dist-test-taskrNqUCn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786292417722232-21710-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786292417722232-21710-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786292417722232-21710-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786292417722232-21710-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.51.188:40535
--webserver_interface=127.21.51.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.188.keytab
--principal=kudu/127.21.51.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.51.148:44317
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.51.190:45837,127.21.51.189:44933,127.21.51.188:40535
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786292417722232-21710-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786292417722232-21710-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786292417722232-21710-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 16:24:55.079494 23641 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.51.189" port: 44933 } attempt: 3
Aug 09 16:24:55 dist-test-slave-s11t krb5kdc[23507](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292694, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.51.190@KRBTEST.COM for kudu/127.21.51.189@KRBTEST.COM
I20260809 16:24:55.087158 23644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:55.079880 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.189:44933 (local address 127.0.0.1:44782)
0809 16:24:55.079938 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0809 16:24:55.079951 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 16:24:55.080022 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:24:55.080193 (+ 171us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:24:55.080194 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:24:55.080371 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:24:55.080451 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:24:55.080452 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:24:55.081192 (+ 740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:24:55.081194 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:24:55.081738 (+ 544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:24:55.081740 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:24:55.082169 (+ 429us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:24:55.082170 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:24:55.082255 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:24:55.082740 (+ 485us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:24:55.082745 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:24:55.083953 (+ 1208us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:24:55.086072 (+ 2119us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:24:55.086073 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:24:55.086074 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:24:55.086260 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:24:55.086477 (+ 217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:24:55.086478 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:24:55.086479 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:24:55.086547 (+ 68us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:24:55.086808 (+ 261us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:24:55.086809 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:24:55.086861 (+ 52us) client_negotiation.cc:769] Sending connection context
0809 16:24:55.086944 (+ 83us) client_negotiation.cc:240] Negotiation successful
0809 16:24:55.087055 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31}
I20260809 16:24:55.087158 23755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:55.079972 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44782 (local address 127.21.51.189:44933)
0809 16:24:55.080046 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0809 16:24:55.080048 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:24:55.080072 (+ 24us) server_negotiation.cc:407] Connection header received
0809 16:24:55.080107 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:24:55.080108 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:24:55.080120 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:24:55.080183 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:24:55.080540 (+ 357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:24:55.081106 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:24:55.081832 (+ 726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:24:55.082087 (+ 255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:24:55.082170 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:24:55.084296 (+ 2126us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:24:55.084298 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:24:55.084298 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:24:55.084306 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:24:55.085954 (+ 1648us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:24:55.086339 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:24:55.086341 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:24:55.086341 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:24:55.086408 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:24:55.086614 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:24:55.086615 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:24:55.086615 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:24:55.086738 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:24:55.086818 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0809 16:24:55.086985 (+ 167us) server_negotiation.cc:299] Negotiation successful
0809 16:24:55.087063 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260809 16:24:55.087524 23641 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.51.188" port: 40535 } has no permanent_uuid. Determining permanent_uuid...
I20260809 16:24:55.087895 23644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:55.087789 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.188:40535 (local address 127.0.0.1:42896)
0809 16:24:55.087842 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0809 16:24:55.087870 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.51.188:40535: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260809 16:24:55.088126 23750 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.51.188" port: 40535 } attempt: 1
I20260809 16:24:55.088336 23644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:55.088249 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.188:40535 (local address 127.0.0.1:42904)
0809 16:24:55.088290 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0809 16:24:55.088313 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.51.188:40535: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
W20260809 16:24:55.088446 23641 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.51.188:40535: Network error: Client connection negotiation failed: client connection to 127.21.51.188:40535: connect: Connection refused (error 111)
I20260809 16:24:55.088569 23753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:55.088382 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.188:40535 (local address 127.0.0.1:42914)
0809 16:24:55.088483 (+ 101us) negotiation.cc:107] Waiting for socket to connect
0809 16:24:55.088537 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.51.188:40535: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":83}
I20260809 16:24:55.088917 23753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:55.088831 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.188:40535 (local address 127.0.0.1:42930)
0809 16:24:55.088879 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0809 16:24:55.088901 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.51.188:40535: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260809 16:24:55.089031 23750 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.51.188:40535: Network error: Client connection negotiation failed: client connection to 127.21.51.188:40535: connect: Connection refused (error 111)
I20260809 16:24:55.119655 23641 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.51.188" port: 40535 } attempt: 1
I20260809 16:24:55.120184 23644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:55.120043 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.188:40535 (local address 127.0.0.1:42940)
0809 16:24:55.120109 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0809 16:24:55.120151 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.51.188:40535: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260809 16:24:55.120666 23644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:55.120569 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.188:40535 (local address 127.0.0.1:42954)
0809 16:24:55.120617 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0809 16:24:55.120646 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.51.188:40535: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260809 16:24:55.120831 23641 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.51.188:40535: Network error: Client connection negotiation failed: client connection to 127.21.51.188:40535: connect: Connection refused (error 111)
W20260809 16:24:55.136478 23761 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260809 16:24:55.136675 23761 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:24:55.136745 23761 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:24:55.136783 23761 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:24:55.137885 23761 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 16:24:55.137919 23761 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:24:55.137933 23761 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:24:55.137946 23761 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 16:24:55.137959 23761 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 16:24:55.139355 23761 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.51.148:44317
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786292417722232-21710-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786292417722232-21710-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.21.51.190:45837,127.21.51.189:44933,127.21.51.188:40535
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.188.keytab
--principal=kudu/127.21.51.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.51.188:40535
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786292417722232-21710-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.21.51.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23761
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786292417722232-21710-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:24:55.139547 23761 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:24:55.139714 23761 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:24:55.144622 23777 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:24:55.144726 23773 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:24:55.144644 23774 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:24:55.144927 23761 server_base.cc:1034] running on GCE node
I20260809 16:24:55.145179 23761 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:24:55.145459 23761 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:24:55.146602 23761 hybrid_clock.cc:648] HybridClock initialized: now 1786292695146580 us; error 40 us; skew 500 ppm
Aug 09 16:24:55 dist-test-slave-s11t krb5kdc[23507](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292695, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.51.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:24:55.149281 23761 init.cc:377] Logged in from keytab as kudu/127.21.51.188@KRBTEST.COM (short username kudu)
I20260809 16:24:55.149791 23761 webserver.cc:460] Webserver started at http://127.21.51.188:46071/ using document root <none> and password file <none>
I20260809 16:24:55.150030 23761 fs_manager.cc:359] Metadata directory not provided
I20260809 16:24:55.150076 23761 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:24:55.150238 23761 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:24:55.151297 23761 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786292417722232-21710-0/minicluster-data/master-2/data/instance:
uuid: "9d61eb883bce4c6eb03609d81a8600f8"
format_stamp: "Formatted at 2026-08-09 16:24:55 on dist-test-slave-s11t"
server_key: "3288e1230e77ac4d51cbd84eef5e69a3"
server_key_iv: "58057a9b11a5ebe0f32ee1440fd1665c"
server_key_version: "encryptionkey@0"
I20260809 16:24:55.151664 23761 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786292417722232-21710-0/minicluster-data/master-2/wal/instance:
uuid: "9d61eb883bce4c6eb03609d81a8600f8"
format_stamp: "Formatted at 2026-08-09 16:24:55 on dist-test-slave-s11t"
server_key: "3288e1230e77ac4d51cbd84eef5e69a3"
server_key_iv: "58057a9b11a5ebe0f32ee1440fd1665c"
server_key_version: "encryptionkey@0"
I20260809 16:24:55.153066 23761 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 16:24:55.153826 23789 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:24:55.154006 23761 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 16:24:55.154098 23761 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786292417722232-21710-0/minicluster-data/master-2/data,/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786292417722232-21710-0/minicluster-data/master-2/wal
uuid: "9d61eb883bce4c6eb03609d81a8600f8"
format_stamp: "Formatted at 2026-08-09 16:24:55 on dist-test-slave-s11t"
server_key: "3288e1230e77ac4d51cbd84eef5e69a3"
server_key_iv: "58057a9b11a5ebe0f32ee1440fd1665c"
server_key_version: "encryptionkey@0"
I20260809 16:24:55.154179 23761 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786292417722232-21710-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786292417722232-21710-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786292417722232-21710-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:24:55.157158 23750 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.51.188" port: 40535 } attempt: 2
I20260809 16:24:55.157588 23753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:55.157430 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.188:40535 (local address 127.0.0.1:42970)
0809 16:24:55.157491 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0809 16:24:55.157546 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.51.188:40535: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260809 16:24:55.157969 23753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:55.157854 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.188:40535 (local address 127.0.0.1:42978)
0809 16:24:55.157898 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0809 16:24:55.157946 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.51.188:40535: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260809 16:24:55.158052 23750 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.51.188:40535: Network error: Client connection negotiation failed: client connection to 127.21.51.188:40535: connect: Connection refused (error 111)
I20260809 16:24:55.167799 23761 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:24:55.170756 23761 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:24:55.170918 23761 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:24:55.174973 23761 rpc_server.cc:307] RPC server started. Bound to: 127.21.51.188:40535
I20260809 16:24:55.175007 23865 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.51.188:40535 every 8 connection(s)
I20260809 16:24:55.175379 23761 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786292417722232-21710-0/minicluster-data/master-2/data/info.pb
I20260809 16:24:55.176415 23866 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:24:55.176577 21710 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrNqUCn/build/release/bin/kudu as pid 23761
I20260809 16:24:55.176687 21710 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786292417722232-21710-0/minicluster-data/master-2/wal/instance
I20260809 16:24:55.176906 21710 external_mini_cluster.cc:1428] Setting key 18a2cb09245d86677be1f264c5744389
I20260809 16:24:55.179003 23866 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.51.190" port: 45837 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 16:24:55 dist-test-slave-s11t krb5kdc[23507](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292694, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.51.188@KRBTEST.COM
Aug 09 16:24:55 dist-test-slave-s11t krb5kdc[23507](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292695, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.51.188@KRBTEST.COM for kudu/127.21.51.190@KRBTEST.COM
I20260809 16:24:55.188359 23870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:55.177460 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42982 (local address 127.21.51.188:40535)
0809 16:24:55.177658 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0809 16:24:55.177668 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 16:24:55.177703 (+ 35us) server_negotiation.cc:407] Connection header received
0809 16:24:55.178209 (+ 506us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:24:55.178218 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:24:55.178285 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:24:55.178450 (+ 165us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:24:55.179052 (+ 602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:24:55.180381 (+ 1329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:24:55.181472 (+ 1091us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:24:55.182069 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:24:55.182181 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:24:55.184581 (+ 2400us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:24:55.184590 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:24:55.184590 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:24:55.184597 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:24:55.186544 (+ 1947us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:24:55.186917 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:24:55.186920 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:24:55.186920 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:24:55.187083 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:24:55.187289 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:24:55.187290 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:24:55.187290 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:24:55.187554 (+ 264us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:24:55.187714 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0809 16:24:55.188013 (+ 299us) server_negotiation.cc:299] Negotiation successful
0809 16:24:55.188146 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":146,"thread_start_us":92,"threads_started":1}
I20260809 16:24:55.188508 23874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:55.179704 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:45837 (local address 127.0.0.1:48490)
0809 16:24:55.179882 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0809 16:24:55.179921 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0809 16:24:55.180000 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:24:55.180138 (+ 138us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:24:55.180140 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:24:55.180486 (+ 346us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:24:55.180568 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:24:55.180570 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:24:55.181482 (+ 912us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:24:55.181483 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:24:55.182331 (+ 848us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:24:55.182333 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:24:55.182808 (+ 475us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:24:55.182809 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:24:55.182865 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:24:55.183333 (+ 468us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:24:55.183341 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:24:55.184927 (+ 1586us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:24:55.187372 (+ 2445us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:24:55.187378 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:24:55.187379 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:24:55.187561 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:24:55.187771 (+ 210us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:24:55.187772 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:24:55.187772 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:24:55.187822 (+ 50us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:24:55.188095 (+ 273us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:24:55.188102 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:24:55.188157 (+ 55us) client_negotiation.cc:769] Sending connection context
0809 16:24:55.188252 (+ 95us) client_negotiation.cc:240] Negotiation successful
0809 16:24:55.188363 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":115,"threads_started":1}
I20260809 16:24:55.188885 23649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:55.179757 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48490 (local address 127.21.51.190:45837)
0809 16:24:55.179828 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0809 16:24:55.179830 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:24:55.179989 (+ 159us) server_negotiation.cc:407] Connection header received
0809 16:24:55.180060 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:24:55.180061 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:24:55.180070 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:24:55.180114 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:24:55.180645 (+ 531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:24:55.181392 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:24:55.182437 (+ 1045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:24:55.182696 (+ 259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:24:55.182773 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:24:55.185018 (+ 2245us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:24:55.185019 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:24:55.185019 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:24:55.185026 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:24:55.187293 (+ 2267us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:24:55.187640 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:24:55.187641 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:24:55.187641 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:24:55.187705 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:24:55.187900 (+ 195us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:24:55.187901 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:24:55.187902 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:24:55.188024 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:24:55.188103 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0809 16:24:55.188293 (+ 190us) server_negotiation.cc:299] Negotiation successful
0809 16:24:55.188369 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260809 16:24:55.189416 23866 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.51.189" port: 44933 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 16:24:55 dist-test-slave-s11t krb5kdc[23507](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292695, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.51.188@KRBTEST.COM for kudu/127.21.51.189@KRBTEST.COM
Aug 09 16:24:55 dist-test-slave-s11t krb5kdc[23507](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292694, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.51.190@KRBTEST.COM
I20260809 16:24:55.195994 23641 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.51.188" port: 40535 } attempt: 2
I20260809 16:24:55.197342 23874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:55.189658 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.189:44933 (local address 127.0.0.1:44792)
0809 16:24:55.189716 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0809 16:24:55.189728 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 16:24:55.189769 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:24:55.189882 (+ 113us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:24:55.189882 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:24:55.190093 (+ 211us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:24:55.190137 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:24:55.190139 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:24:55.191066 (+ 927us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:24:55.191067 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:24:55.191828 (+ 761us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:24:55.191830 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:24:55.192325 (+ 495us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:24:55.192326 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:24:55.192388 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:24:55.192956 (+ 568us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:24:55.192961 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:24:55.194305 (+ 1344us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:24:55.196109 (+ 1804us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:24:55.196110 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:24:55.196111 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:24:55.196339 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:24:55.196580 (+ 241us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:24:55.196581 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:24:55.196581 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:24:55.196651 (+ 70us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:24:55.196990 (+ 339us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:24:55.196991 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:24:55.197056 (+ 65us) client_negotiation.cc:769] Sending connection context
0809 16:24:55.197131 (+ 75us) client_negotiation.cc:240] Negotiation successful
0809 16:24:55.197220 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38,"mutex_wait_us":3}
I20260809 16:24:55.197340 23755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:55.189741 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44792 (local address 127.21.51.189:44933)
0809 16:24:55.189794 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0809 16:24:55.189796 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:24:55.189803 (+ 7us) server_negotiation.cc:407] Connection header received
0809 16:24:55.189824 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:24:55.189825 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:24:55.189833 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:24:55.189875 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:24:55.190218 (+ 343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:24:55.190967 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:24:55.191936 (+ 969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:24:55.192246 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:24:55.192313 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:24:55.194393 (+ 2080us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:24:55.194394 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:24:55.194394 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:24:55.194402 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:24:55.196028 (+ 1626us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:24:55.196414 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:24:55.196416 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:24:55.196416 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:24:55.196485 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:24:55.196748 (+ 263us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:24:55.196749 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:24:55.196750 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:24:55.196898 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:24:55.196959 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0809 16:24:55.197168 (+ 209us) server_negotiation.cc:299] Negotiation successful
0809 16:24:55.197247 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260809 16:24:55.197978 23866 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.51.188" port: 40535 } has no permanent_uuid. Determining permanent_uuid...
I20260809 16:24:55.199002 23649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:55.189311 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48504 (local address 127.21.51.190:45837)
0809 16:24:55.189359 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0809 16:24:55.189361 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:24:55.189428 (+ 67us) server_negotiation.cc:407] Connection header received
0809 16:24:55.189494 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:24:55.189495 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:24:55.189505 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:24:55.189556 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:24:55.190312 (+ 756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:24:55.190990 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:24:55.191940 (+ 950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:24:55.192246 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:24:55.192313 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:24:55.195016 (+ 2703us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:24:55.195017 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:24:55.195018 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:24:55.195024 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:24:55.196979 (+ 1955us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:24:55.197358 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:24:55.197360 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:24:55.197360 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:24:55.197508 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:24:55.197723 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:24:55.197724 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:24:55.197724 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:24:55.197887 (+ 163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:24:55.198334 (+ 447us) server_negotiation.cc:1035] Waiting for connection context
0809 16:24:55.198815 (+ 481us) server_negotiation.cc:299] Negotiation successful
0809 16:24:55.198891 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":38,"server-negotiator.queue_time_us":16}
Aug 09 16:24:55 dist-test-slave-s11t krb5kdc[23507](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292694, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.51.190@KRBTEST.COM for kudu/127.21.51.188@KRBTEST.COM
Aug 09 16:24:55 dist-test-slave-s11t krb5kdc[23507](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292695, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.51.188@KRBTEST.COM for kudu/127.21.51.188@KRBTEST.COM
I20260809 16:24:55.204119 23644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:55.196240 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.188:40535 (local address 127.0.0.1:42996)
0809 16:24:55.196291 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0809 16:24:55.196301 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 16:24:55.196335 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:24:55.196543 (+ 208us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:24:55.196544 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:24:55.196801 (+ 257us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:24:55.196840 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:24:55.196841 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:24:55.197783 (+ 942us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:24:55.197784 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:24:55.198558 (+ 774us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:24:55.198561 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:24:55.199044 (+ 483us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:24:55.199046 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:24:55.199112 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:24:55.199590 (+ 478us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:24:55.199595 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:24:55.200900 (+ 1305us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:24:55.202964 (+ 2064us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:24:55.202965 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:24:55.202966 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:24:55.203182 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:24:55.203392 (+ 210us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:24:55.203394 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:24:55.203394 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:24:55.203467 (+ 73us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:24:55.203795 (+ 328us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:24:55.203796 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:24:55.203848 (+ 52us) client_negotiation.cc:769] Sending connection context
0809 16:24:55.203913 (+ 65us) client_negotiation.cc:240] Negotiation successful
0809 16:24:55.204029 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36}
I20260809 16:24:55.204492 23870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:55.196409 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42996 (local address 127.21.51.188:40535)
0809 16:24:55.196444 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0809 16:24:55.196446 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:24:55.196451 (+ 5us) server_negotiation.cc:407] Connection header received
0809 16:24:55.196467 (+ 16us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:24:55.196467 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:24:55.196475 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:24:55.196524 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:24:55.196924 (+ 400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:24:55.197697 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:24:55.198642 (+ 945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:24:55.198961 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:24:55.199036 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:24:55.200988 (+ 1952us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:24:55.200989 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:24:55.200989 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:24:55.200996 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:24:55.202889 (+ 1893us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:24:55.203271 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:24:55.203272 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:24:55.203272 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:24:55.203331 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:24:55.203555 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:24:55.203557 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:24:55.203557 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:24:55.203702 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:24:55.203766 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0809 16:24:55.204336 (+ 570us) server_negotiation.cc:299] Negotiation successful
0809 16:24:55.204411 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":15,"server-negotiator.queue_time_us":20}
Aug 09 16:24:55 dist-test-slave-s11t krb5kdc[23507](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292694, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.51.189@KRBTEST.COM
I20260809 16:24:55.206295 23641 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9a8d69c13e9c4d919b2fe59e4cefe39a: Bootstrap starting.
I20260809 16:24:55.206429 23874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:55.198201 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.188:40535 (local address 127.0.0.1:43004)
0809 16:24:55.198362 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0809 16:24:55.198371 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 16:24:55.198403 (+ 32us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:24:55.198548 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:24:55.198549 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:24:55.198686 (+ 137us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:24:55.198707 (+ 21us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:24:55.198709 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:24:55.200393 (+ 1684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:24:55.200394 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:24:55.201093 (+ 699us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:24:55.201095 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:24:55.201607 (+ 512us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:24:55.201607 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:24:55.201659 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:24:55.202049 (+ 390us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:24:55.202055 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:24:55.203439 (+ 1384us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:24:55.205357 (+ 1918us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:24:55.205359 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:24:55.205359 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:24:55.205525 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:24:55.205725 (+ 200us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:24:55.205726 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:24:55.205726 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:24:55.205779 (+ 53us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:24:55.206114 (+ 335us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:24:55.206115 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:24:55.206161 (+ 46us) client_negotiation.cc:769] Sending connection context
0809 16:24:55.206245 (+ 84us) client_negotiation.cc:240] Negotiation successful
0809 16:24:55.206347 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148}
I20260809 16:24:55.206640 23875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:55.198260 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43004 (local address 127.21.51.188:40535)
0809 16:24:55.198414 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0809 16:24:55.198416 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:24:55.198423 (+ 7us) server_negotiation.cc:407] Connection header received
0809 16:24:55.198456 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:24:55.198456 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:24:55.198464 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:24:55.198546 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:24:55.198889 (+ 343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:24:55.200297 (+ 1408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:24:55.201182 (+ 885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:24:55.201530 (+ 348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:24:55.201607 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:24:55.203508 (+ 1901us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:24:55.203509 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:24:55.203509 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:24:55.203515 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:24:55.205282 (+ 1767us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:24:55.205596 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:24:55.205598 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:24:55.205598 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:24:55.205650 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:24:55.205850 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:24:55.205851 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:24:55.205852 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:24:55.206011 (+ 159us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:24:55.206083 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0809 16:24:55.206296 (+ 213us) server_negotiation.cc:299] Negotiation successful
0809 16:24:55.206439 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":29,"server-negotiator.queue_time_us":134,"thread_start_us":108,"threads_started":1}
I20260809 16:24:55.207077 23641 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9a8d69c13e9c4d919b2fe59e4cefe39a: Neither blocks nor log segments found. Creating new log.
I20260809 16:24:55.207357 23641 log.cc:824] T 00000000000000000000000000000000 P 9a8d69c13e9c4d919b2fe59e4cefe39a: Log is configured to *not* fsync() on all Append() calls
I20260809 16:24:55.208243 23755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:55.199865 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44802 (local address 127.21.51.189:44933)
0809 16:24:55.199899 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0809 16:24:55.199901 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:24:55.199910 (+ 9us) server_negotiation.cc:407] Connection header received
0809 16:24:55.199931 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:24:55.199932 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:24:55.199940 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:24:55.200011 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:24:55.200573 (+ 562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:24:55.201239 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:24:55.202119 (+ 880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:24:55.202337 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:24:55.202414 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:24:55.205168 (+ 2754us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:24:55.205169 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:24:55.205170 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:24:55.205177 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:24:55.206897 (+ 1720us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:24:55.207255 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:24:55.207256 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:24:55.207256 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:24:55.207310 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:24:55.207524 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:24:55.207525 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:24:55.207526 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:24:55.207654 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:24:55.207815 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0809 16:24:55.208001 (+ 186us) server_negotiation.cc:299] Negotiation successful
0809 16:24:55.208074 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260809 16:24:55.208346 23641 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9a8d69c13e9c4d919b2fe59e4cefe39a: No bootstrap required, opened a new log
I20260809 16:24:55.208950 23866 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9d61eb883bce4c6eb03609d81a8600f8: Bootstrap starting.
I20260809 16:24:55.209633 23866 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9d61eb883bce4c6eb03609d81a8600f8: Neither blocks nor log segments found. Creating new log.
I20260809 16:24:55.209635 21710 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 16:24:55.209983 23866 log.cc:824] T 00000000000000000000000000000000 P 9d61eb883bce4c6eb03609d81a8600f8: Log is configured to *not* fsync() on all Append() calls
I20260809 16:24:55.210816 23866 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9d61eb883bce4c6eb03609d81a8600f8: No bootstrap required, opened a new log
I20260809 16:24:55.210716 23641 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9a8d69c13e9c4d919b2fe59e4cefe39a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a8d69c13e9c4d919b2fe59e4cefe39a" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 45837 } } peers { permanent_uuid: "4fc0017b5efd45aebd25c4c6a6ac994b" member_type: VOTER last_known_addr { host: "127.21.51.189" port: 44933 } } peers { permanent_uuid: "9d61eb883bce4c6eb03609d81a8600f8" member_type: VOTER last_known_addr { host: "127.21.51.188" port: 40535 } }
I20260809 16:24:55.210791 23593 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "9a8d69c13e9c4d919b2fe59e4cefe39a"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48504
I20260809 16:24:55.210867 23641 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9a8d69c13e9c4d919b2fe59e4cefe39a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9a8d69c13e9c4d919b2fe59e4cefe39a, State: Initialized, Role: FOLLOWER
I20260809 16:24:55.210958 23641 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9a8d69c13e9c4d919b2fe59e4cefe39a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a8d69c13e9c4d919b2fe59e4cefe39a" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 45837 } } peers { permanent_uuid: "4fc0017b5efd45aebd25c4c6a6ac994b" member_type: VOTER last_known_addr { host: "127.21.51.189" port: 44933 } } peers { permanent_uuid: "9d61eb883bce4c6eb03609d81a8600f8" member_type: VOTER last_known_addr { host: "127.21.51.188" port: 40535 } }
I20260809 16:24:55.211256 23881 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9a8d69c13e9c4d919b2fe59e4cefe39a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a8d69c13e9c4d919b2fe59e4cefe39a" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 45837 } } peers { permanent_uuid: "4fc0017b5efd45aebd25c4c6a6ac994b" member_type: VOTER last_known_addr { host: "127.21.51.189" port: 44933 } } peers { permanent_uuid: "9d61eb883bce4c6eb03609d81a8600f8" member_type: VOTER last_known_addr { host: "127.21.51.188" port: 40535 } } }
I20260809 16:24:55.211297 23641 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9a8d69c13e9c4d919b2fe59e4cefe39a [sys.catalog]: configured and running, proceeding with master startup.
I20260809 16:24:55.211359 23593 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9a8d69c13e9c4d919b2fe59e4cefe39a [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260809 16:24:55.211428 23593 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9a8d69c13e9c4d919b2fe59e4cefe39a [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:24:55.212253 23593 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9a8d69c13e9c4d919b2fe59e4cefe39a [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a8d69c13e9c4d919b2fe59e4cefe39a" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 45837 } } peers { permanent_uuid: "4fc0017b5efd45aebd25c4c6a6ac994b" member_type: VOTER last_known_addr { host: "127.21.51.189" port: 44933 } } peers { permanent_uuid: "9d61eb883bce4c6eb03609d81a8600f8" member_type: VOTER last_known_addr { host: "127.21.51.188" port: 40535 } }
I20260809 16:24:55.212505 23593 leader_election.cc:288] T 00000000000000000000000000000000 P 9a8d69c13e9c4d919b2fe59e4cefe39a [CANDIDATE]: Term 1 election: Requested vote from peers 4fc0017b5efd45aebd25c4c6a6ac994b (127.21.51.189:44933), 9d61eb883bce4c6eb03609d81a8600f8 (127.21.51.188:40535)
I20260809 16:24:55.211354 23881 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9a8d69c13e9c4d919b2fe59e4cefe39a [sys.catalog]: This master's current role is: FOLLOWER
I20260809 16:24:55.213150 23866 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9d61eb883bce4c6eb03609d81a8600f8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a8d69c13e9c4d919b2fe59e4cefe39a" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 45837 } } peers { permanent_uuid: "4fc0017b5efd45aebd25c4c6a6ac994b" member_type: VOTER last_known_addr { host: "127.21.51.189" port: 44933 } } peers { permanent_uuid: "9d61eb883bce4c6eb03609d81a8600f8" member_type: VOTER last_known_addr { host: "127.21.51.188" port: 40535 } }
I20260809 16:24:55.213311 23866 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9d61eb883bce4c6eb03609d81a8600f8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9d61eb883bce4c6eb03609d81a8600f8, State: Initialized, Role: FOLLOWER
I20260809 16:24:55.213392 23866 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9d61eb883bce4c6eb03609d81a8600f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a8d69c13e9c4d919b2fe59e4cefe39a" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 45837 } } peers { permanent_uuid: "4fc0017b5efd45aebd25c4c6a6ac994b" member_type: VOTER last_known_addr { host: "127.21.51.189" port: 44933 } } peers { permanent_uuid: "9d61eb883bce4c6eb03609d81a8600f8" member_type: VOTER last_known_addr { host: "127.21.51.188" port: 40535 } }
I20260809 16:24:55.213719 23866 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9d61eb883bce4c6eb03609d81a8600f8 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 16:24:55.214254 23828 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "9a8d69c13e9c4d919b2fe59e4cefe39a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "9d61eb883bce4c6eb03609d81a8600f8"
I20260809 16:24:55.214396 23828 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9d61eb883bce4c6eb03609d81a8600f8 [term 0 FOLLOWER]: Advancing to term 1
W20260809 16:24:55.214557 23901 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 9a8d69c13e9c4d919b2fe59e4cefe39a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 16:24:55.214613 23901 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 16:24:55.215739 23828 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 9d61eb883bce4c6eb03609d81a8600f8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9a8d69c13e9c4d919b2fe59e4cefe39a in term 1.
I20260809 16:24:55.215987 23553 leader_election.cc:302] T 00000000000000000000000000000000 P 9a8d69c13e9c4d919b2fe59e4cefe39a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9a8d69c13e9c4d919b2fe59e4cefe39a, 9d61eb883bce4c6eb03609d81a8600f8; no voters:
I20260809 16:24:55.216109 23881 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9a8d69c13e9c4d919b2fe59e4cefe39a [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:24:55.216238 23881 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9a8d69c13e9c4d919b2fe59e4cefe39a [term 1 LEADER]: Becoming Leader. State: Replica: 9a8d69c13e9c4d919b2fe59e4cefe39a, State: Running, Role: LEADER
I20260809 16:24:55.216323 23881 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9a8d69c13e9c4d919b2fe59e4cefe39a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a8d69c13e9c4d919b2fe59e4cefe39a" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 45837 } } peers { permanent_uuid: "4fc0017b5efd45aebd25c4c6a6ac994b" member_type: VOTER last_known_addr { host: "127.21.51.189" port: 44933 } } peers { permanent_uuid: "9d61eb883bce4c6eb03609d81a8600f8" member_type: VOTER last_known_addr { host: "127.21.51.188" port: 40535 } }
I20260809 16:24:55.216727 23881 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9a8d69c13e9c4d919b2fe59e4cefe39a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9a8d69c13e9c4d919b2fe59e4cefe39a. Latest consensus state: current_term: 1 leader_uuid: "9a8d69c13e9c4d919b2fe59e4cefe39a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a8d69c13e9c4d919b2fe59e4cefe39a" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 45837 } } peers { permanent_uuid: "4fc0017b5efd45aebd25c4c6a6ac994b" member_type: VOTER last_known_addr { host: "127.21.51.189" port: 44933 } } peers { permanent_uuid: "9d61eb883bce4c6eb03609d81a8600f8" member_type: VOTER last_known_addr { host: "127.21.51.188" port: 40535 } } }
I20260809 16:24:55.216799 23881 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9a8d69c13e9c4d919b2fe59e4cefe39a [sys.catalog]: This master's current role is: LEADER
I20260809 16:24:55.216141 23527 master_runner.cc:186] Error getting master registration for 127.21.51.189:44933: OK, instance_id { permanent_uuid: "4fc0017b5efd45aebd25c4c6a6ac994b" instance_seqno: 1786292695014603 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 16:24:55.218538 23527 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260809 16:24:55.219250 23933 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 9d61eb883bce4c6eb03609d81a8600f8: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 16:24:55.219368 23933 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 16:24:55.218777 23761 master_runner.cc:186] Error getting master registration for 127.21.51.189:44933: OK, instance_id { permanent_uuid: "4fc0017b5efd45aebd25c4c6a6ac994b" instance_seqno: 1786292695014603 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 16:24:55.221340 23761 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260809 16:24:55.222534 23937 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 16:24:55.222769 23937 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 16:24:55.223841 23875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:55.213964 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43018 (local address 127.21.51.188:40535)
0809 16:24:55.214004 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0809 16:24:55.214006 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:24:55.215120 (+ 1114us) server_negotiation.cc:407] Connection header received
0809 16:24:55.215229 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:24:55.215230 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:24:55.215242 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:24:55.215274 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:24:55.215875 (+ 601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:24:55.216610 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:24:55.217769 (+ 1159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:24:55.218072 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:24:55.218114 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:24:55.219752 (+ 1638us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:24:55.219753 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:24:55.219754 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:24:55.219760 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:24:55.221442 (+ 1682us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:24:55.221807 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:24:55.221823 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:24:55.221824 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:24:55.221884 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:24:55.222463 (+ 579us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:24:55.222464 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:24:55.222465 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:24:55.222618 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:24:55.222729 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0809 16:24:55.223662 (+ 933us) server_negotiation.cc:299] Negotiation successful
0809 16:24:55.223749 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260809 16:24:55.224673 23828 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 9d61eb883bce4c6eb03609d81a8600f8 [term 1 FOLLOWER]: Refusing update from remote peer 9a8d69c13e9c4d919b2fe59e4cefe39a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 16:24:55.224874 23910 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9d61eb883bce4c6eb03609d81a8600f8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a8d69c13e9c4d919b2fe59e4cefe39a" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 45837 } } peers { permanent_uuid: "4fc0017b5efd45aebd25c4c6a6ac994b" member_type: VOTER last_known_addr { host: "127.21.51.189" port: 44933 } } peers { permanent_uuid: "9d61eb883bce4c6eb03609d81a8600f8" member_type: VOTER last_known_addr { host: "127.21.51.188" port: 40535 } } }
I20260809 16:24:55.224974 23910 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9d61eb883bce4c6eb03609d81a8600f8 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 16:24:55.225590 23932 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 9a8d69c13e9c4d919b2fe59e4cefe39a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9d61eb883bce4c6eb03609d81a8600f8" member_type: VOTER last_known_addr { host: "127.21.51.188" port: 40535 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260809 16:24:55.227643 23910 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9d61eb883bce4c6eb03609d81a8600f8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9a8d69c13e9c4d919b2fe59e4cefe39a. Latest consensus state: current_term: 1 leader_uuid: "9a8d69c13e9c4d919b2fe59e4cefe39a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a8d69c13e9c4d919b2fe59e4cefe39a" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 45837 } } peers { permanent_uuid: "4fc0017b5efd45aebd25c4c6a6ac994b" member_type: VOTER last_known_addr { host: "127.21.51.189" port: 44933 } } peers { permanent_uuid: "9d61eb883bce4c6eb03609d81a8600f8" member_type: VOTER last_known_addr { host: "127.21.51.188" port: 40535 } } }
I20260809 16:24:55.227737 23910 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9d61eb883bce4c6eb03609d81a8600f8 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 16:24:55.228731 23932 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9a8d69c13e9c4d919b2fe59e4cefe39a [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "9a8d69c13e9c4d919b2fe59e4cefe39a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a8d69c13e9c4d919b2fe59e4cefe39a" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 45837 } } peers { permanent_uuid: "4fc0017b5efd45aebd25c4c6a6ac994b" member_type: VOTER last_known_addr { host: "127.21.51.189" port: 44933 } } peers { permanent_uuid: "9d61eb883bce4c6eb03609d81a8600f8" member_type: VOTER last_known_addr { host: "127.21.51.188" port: 40535 } } }
I20260809 16:24:55.228830 23932 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9a8d69c13e9c4d919b2fe59e4cefe39a [sys.catalog]: This master's current role is: LEADER
I20260809 16:24:55.229010 23940 mvcc.cc:205] Tried to move back new op lower bound from 7316654879636828160 to 7316654879606669312. Current Snapshot: MvccSnapshot[applied={T|T < 7316654879636828160}]
I20260809 16:24:55.229493 23910 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9d61eb883bce4c6eb03609d81a8600f8 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "9a8d69c13e9c4d919b2fe59e4cefe39a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a8d69c13e9c4d919b2fe59e4cefe39a" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 45837 } } peers { permanent_uuid: "4fc0017b5efd45aebd25c4c6a6ac994b" member_type: VOTER last_known_addr { host: "127.21.51.189" port: 44933 } } peers { permanent_uuid: "9d61eb883bce4c6eb03609d81a8600f8" member_type: VOTER last_known_addr { host: "127.21.51.188" port: 40535 } } }
I20260809 16:24:55.229712 23910 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9d61eb883bce4c6eb03609d81a8600f8 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 16:24:55.231647 23941 mvcc.cc:205] Tried to move back new op lower bound from 7316654879636828160 to 7316654879606669312. Current Snapshot: MvccSnapshot[applied={T|T < 7316654879636828160}]
I20260809 16:24:55.234217 23649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:55.214852 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48516 (local address 127.21.51.190:45837)
0809 16:24:55.214894 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0809 16:24:55.214896 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:24:55.224991 (+ 10095us) server_negotiation.cc:407] Connection header received
0809 16:24:55.225091 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:24:55.225093 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:24:55.225103 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:24:55.225129 (+ 26us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:24:55.225874 (+ 745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:24:55.226618 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:24:55.228901 (+ 2283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:24:55.229206 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:24:55.229252 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:24:55.231078 (+ 1826us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:24:55.231079 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:24:55.231080 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:24:55.231087 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:24:55.232876 (+ 1789us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:24:55.233260 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:24:55.233261 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:24:55.233262 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:24:55.233316 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:24:55.233581 (+ 265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:24:55.233582 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:24:55.233583 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:24:55.233710 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:24:55.233822 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0809 16:24:55.234038 (+ 216us) server_negotiation.cc:299] Negotiation successful
0809 16:24:55.234111 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260809 16:24:55.237389 23937 catalog_manager.cc:1347] Generated new cluster ID: 00d596e2e7b84385830a86b3048d2204
I20260809 16:24:55.237443 23937 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 16:24:55.247507 23755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:55.215247 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44804 (local address 127.21.51.189:44933)
0809 16:24:55.215299 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0809 16:24:55.215302 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 16:24:55.225159 (+ 9857us) server_negotiation.cc:407] Connection header received
0809 16:24:55.225232 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:24:55.225233 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:24:55.225256 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:24:55.225309 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:24:55.228507 (+ 3198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:24:55.229240 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:24:55.236992 (+ 7752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:24:55.237318 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:24:55.237389 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:24:55.244447 (+ 7058us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:24:55.244448 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:24:55.244448 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:24:55.244456 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:24:55.246145 (+ 1689us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:24:55.246616 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:24:55.246617 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:24:55.246618 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:24:55.246680 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:24:55.246898 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:24:55.246899 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:24:55.246899 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:24:55.247032 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:24:55.247122 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0809 16:24:55.247299 (+ 177us) server_negotiation.cc:299] Negotiation successful
0809 16:24:55.247376 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260809 16:24:55.249163 23750 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.51.188" port: 40535 } attempt: 3
Aug 09 16:24:55 dist-test-slave-s11t krb5kdc[23507](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292694, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.51.189@KRBTEST.COM for kudu/127.21.51.188@KRBTEST.COM
I20260809 16:24:55.257165 23753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:55.249422 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.188:40535 (local address 127.0.0.1:43020)
0809 16:24:55.249482 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0809 16:24:55.249492 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 16:24:55.249536 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:24:55.249760 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:24:55.249761 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:24:55.249922 (+ 161us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:24:55.249957 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:24:55.249958 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:24:55.250861 (+ 903us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:24:55.250863 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:24:55.251421 (+ 558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:24:55.251423 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:24:55.251873 (+ 450us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:24:55.251874 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:24:55.251933 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:24:55.252339 (+ 406us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:24:55.252345 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:24:55.253935 (+ 1590us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:24:55.255608 (+ 1673us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:24:55.255609 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:24:55.255609 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:24:55.255837 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:24:55.256338 (+ 501us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:24:55.256339 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:24:55.256339 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:24:55.256412 (+ 73us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:24:55.256836 (+ 424us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:24:55.256837 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:24:55.256882 (+ 45us) client_negotiation.cc:769] Sending connection context
0809 16:24:55.256971 (+ 89us) client_negotiation.cc:240] Negotiation successful
0809 16:24:55.257069 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42}
I20260809 16:24:55.257915 23875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:55.249596 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43020 (local address 127.21.51.188:40535)
0809 16:24:55.249660 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0809 16:24:55.249662 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:24:55.249671 (+ 9us) server_negotiation.cc:407] Connection header received
0809 16:24:55.249693 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:24:55.249694 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:24:55.249705 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:24:55.249758 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:24:55.250048 (+ 290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:24:55.250786 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:24:55.251507 (+ 721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:24:55.251813 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:24:55.251875 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:24:55.254029 (+ 2154us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:24:55.254030 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:24:55.254030 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:24:55.254037 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:24:55.255503 (+ 1466us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:24:55.255903 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:24:55.255905 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:24:55.255905 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:24:55.255972 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:24:55.256503 (+ 531us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:24:55.256504 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:24:55.256504 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:24:55.256731 (+ 227us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:24:55.256809 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0809 16:24:55.257689 (+ 880us) server_negotiation.cc:299] Negotiation successful
0809 16:24:55.257809 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260809 16:24:55.259681 23750 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4fc0017b5efd45aebd25c4c6a6ac994b: Bootstrap starting.
I20260809 16:24:55.260447 23750 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4fc0017b5efd45aebd25c4c6a6ac994b: Neither blocks nor log segments found. Creating new log.
I20260809 16:24:55.260782 23750 log.cc:824] T 00000000000000000000000000000000 P 4fc0017b5efd45aebd25c4c6a6ac994b: Log is configured to *not* fsync() on all Append() calls
I20260809 16:24:55.269496 23750 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4fc0017b5efd45aebd25c4c6a6ac994b: No bootstrap required, opened a new log
I20260809 16:24:55.271358 23750 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4fc0017b5efd45aebd25c4c6a6ac994b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a8d69c13e9c4d919b2fe59e4cefe39a" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 45837 } } peers { permanent_uuid: "4fc0017b5efd45aebd25c4c6a6ac994b" member_type: VOTER last_known_addr { host: "127.21.51.189" port: 44933 } } peers { permanent_uuid: "9d61eb883bce4c6eb03609d81a8600f8" member_type: VOTER last_known_addr { host: "127.21.51.188" port: 40535 } }
I20260809 16:24:55.271481 23750 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4fc0017b5efd45aebd25c4c6a6ac994b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4fc0017b5efd45aebd25c4c6a6ac994b, State: Initialized, Role: FOLLOWER
I20260809 16:24:55.271551 23750 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4fc0017b5efd45aebd25c4c6a6ac994b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a8d69c13e9c4d919b2fe59e4cefe39a" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 45837 } } peers { permanent_uuid: "4fc0017b5efd45aebd25c4c6a6ac994b" member_type: VOTER last_known_addr { host: "127.21.51.189" port: 44933 } } peers { permanent_uuid: "9d61eb883bce4c6eb03609d81a8600f8" member_type: VOTER last_known_addr { host: "127.21.51.188" port: 40535 } }
I20260809 16:24:55.271852 23750 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4fc0017b5efd45aebd25c4c6a6ac994b [sys.catalog]: configured and running, proceeding with master startup.
I20260809 16:24:55.272943 23710 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4fc0017b5efd45aebd25c4c6a6ac994b [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:24:55.273790 23710 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 4fc0017b5efd45aebd25c4c6a6ac994b [term 1 FOLLOWER]: Refusing update from remote peer 9a8d69c13e9c4d919b2fe59e4cefe39a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 16:24:55.274098 23881 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 9a8d69c13e9c4d919b2fe59e4cefe39a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4fc0017b5efd45aebd25c4c6a6ac994b" member_type: VOTER last_known_addr { host: "127.21.51.189" port: 44933 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 16:24:55.275467 23712 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "9a8d69c13e9c4d919b2fe59e4cefe39a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "4fc0017b5efd45aebd25c4c6a6ac994b"
W20260809 16:24:55.276969 23971 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 4fc0017b5efd45aebd25c4c6a6ac994b: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 16:24:55.277130 23971 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 16:24:55.277594 23959 mvcc.cc:205] Tried to move back new op lower bound from 7316654879636828160 to 7316654879606669312. Current Snapshot: MvccSnapshot[applied={T|T < 7316654879636828160}]
I20260809 16:24:55.277822 23958 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4fc0017b5efd45aebd25c4c6a6ac994b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9a8d69c13e9c4d919b2fe59e4cefe39a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a8d69c13e9c4d919b2fe59e4cefe39a" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 45837 } } peers { permanent_uuid: "4fc0017b5efd45aebd25c4c6a6ac994b" member_type: VOTER last_known_addr { host: "127.21.51.189" port: 44933 } } peers { permanent_uuid: "9d61eb883bce4c6eb03609d81a8600f8" member_type: VOTER last_known_addr { host: "127.21.51.188" port: 40535 } } }
I20260809 16:24:55.277936 23958 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4fc0017b5efd45aebd25c4c6a6ac994b [sys.catalog]: This master's current role is: FOLLOWER
I20260809 16:24:55.278012 23958 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4fc0017b5efd45aebd25c4c6a6ac994b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9a8d69c13e9c4d919b2fe59e4cefe39a. Latest consensus state: current_term: 1 leader_uuid: "9a8d69c13e9c4d919b2fe59e4cefe39a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a8d69c13e9c4d919b2fe59e4cefe39a" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 45837 } } peers { permanent_uuid: "4fc0017b5efd45aebd25c4c6a6ac994b" member_type: VOTER last_known_addr { host: "127.21.51.189" port: 44933 } } peers { permanent_uuid: "9d61eb883bce4c6eb03609d81a8600f8" member_type: VOTER last_known_addr { host: "127.21.51.188" port: 40535 } } }
I20260809 16:24:55.278090 23958 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4fc0017b5efd45aebd25c4c6a6ac994b [sys.catalog]: This master's current role is: FOLLOWER
I20260809 16:24:55.278152 23958 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4fc0017b5efd45aebd25c4c6a6ac994b [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "9a8d69c13e9c4d919b2fe59e4cefe39a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a8d69c13e9c4d919b2fe59e4cefe39a" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 45837 } } peers { permanent_uuid: "4fc0017b5efd45aebd25c4c6a6ac994b" member_type: VOTER last_known_addr { host: "127.21.51.189" port: 44933 } } peers { permanent_uuid: "9d61eb883bce4c6eb03609d81a8600f8" member_type: VOTER last_known_addr { host: "127.21.51.188" port: 40535 } } }
I20260809 16:24:55.278980 23958 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4fc0017b5efd45aebd25c4c6a6ac994b [sys.catalog]: This master's current role is: FOLLOWER
I20260809 16:24:55.279716 23712 raft_consensus.cc:2396] T 00000000000000000000000000000000 P 4fc0017b5efd45aebd25c4c6a6ac994b [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 9a8d69c13e9c4d919b2fe59e4cefe39a for term 1 because replica has last-logged OpId of term: 1 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 0 index: 0.
I20260809 16:24:55.286109 23881 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9a8d69c13e9c4d919b2fe59e4cefe39a [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "9a8d69c13e9c4d919b2fe59e4cefe39a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a8d69c13e9c4d919b2fe59e4cefe39a" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 45837 } } peers { permanent_uuid: "4fc0017b5efd45aebd25c4c6a6ac994b" member_type: VOTER last_known_addr { host: "127.21.51.189" port: 44933 } } peers { permanent_uuid: "9d61eb883bce4c6eb03609d81a8600f8" member_type: VOTER last_known_addr { host: "127.21.51.188" port: 40535 } } }
I20260809 16:24:55.286218 23881 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9a8d69c13e9c4d919b2fe59e4cefe39a [sys.catalog]: This master's current role is: LEADER
I20260809 16:24:55.288767 23937 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 16:24:55.289542 23937 catalog_manager.cc:1504] Loading token signing keys...
I20260809 16:24:55.303231 23937 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9a8d69c13e9c4d919b2fe59e4cefe39a: Generated new TSK 0
I20260809 16:24:55.303505 23937 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260809 16:24:55.331888 23649 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:24:55.330251 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48522 (local address 127.21.51.190:45837)
0809 16:24:55.330299 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0809 16:24:55.330301 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:24:55.331242 (+ 941us) server_negotiation.cc:407] Connection header received
0809 16:24:55.331350 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:24:55.331351 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:24:55.331363 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:24:55.331394 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:24:55.331821 (+ 427us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48522: BlockingRecv error: recv got EOF from 127.0.0.1:48522 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
W20260809 16:24:55.332789 23875 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:24:55.330632 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43022 (local address 127.21.51.188:40535)
0809 16:24:55.330672 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0809 16:24:55.330674 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:24:55.332267 (+ 1593us) server_negotiation.cc:407] Connection header received
0809 16:24:55.332361 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:24:55.332362 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:24:55.332373 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:24:55.332401 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:24:55.332727 (+ 326us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43022: BlockingRecv error: recv got EOF from 127.0.0.1:43022 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260809 16:24:55.333612 23755 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:24:55.330980 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44820 (local address 127.21.51.189:44933)
0809 16:24:55.331018 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0809 16:24:55.331020 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:24:55.333103 (+ 2083us) server_negotiation.cc:407] Connection header received
0809 16:24:55.333203 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:24:55.333204 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:24:55.333221 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:24:55.333251 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:24:55.333553 (+ 302us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44820: BlockingRecv error: recv got EOF from 127.0.0.1:44820 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260809 16:24:55.335619 23755 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:24:55.334925 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44832 (local address 127.21.51.189:44933)
0809 16:24:55.334958 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0809 16:24:55.334960 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:24:55.335134 (+ 174us) server_negotiation.cc:407] Connection header received
0809 16:24:55.335228 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:24:55.335229 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:24:55.335246 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:24:55.335272 (+ 26us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:24:55.335576 (+ 304us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44832: BlockingRecv error: recv got EOF from 127.0.0.1:44832 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260809 16:24:55.338666 23755 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:24:55.337974 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44848 (local address 127.21.51.189:44933)
0809 16:24:55.338006 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0809 16:24:55.338008 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:24:55.338201 (+ 193us) server_negotiation.cc:407] Connection header received
0809 16:24:55.338291 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:24:55.338293 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:24:55.338302 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:24:55.338329 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:24:55.338625 (+ 296us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44848: BlockingRecv error: recv got EOF from 127.0.0.1:44848 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
W20260809 16:24:55.342501 23755 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:24:55.341800 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44850 (local address 127.21.51.189:44933)
0809 16:24:55.341833 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0809 16:24:55.341835 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:24:55.342040 (+ 205us) server_negotiation.cc:407] Connection header received
0809 16:24:55.342131 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:24:55.342132 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:24:55.342141 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:24:55.342168 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:24:55.342458 (+ 290us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44850: BlockingRecv error: recv got EOF from 127.0.0.1:44850 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260809 16:24:55.349756 23755 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:24:55.348983 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44854 (local address 127.21.51.189:44933)
0809 16:24:55.349019 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0809 16:24:55.349021 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:24:55.349242 (+ 221us) server_negotiation.cc:407] Connection header received
0809 16:24:55.349338 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:24:55.349339 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:24:55.349349 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:24:55.349377 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:24:55.349708 (+ 331us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44854: BlockingRecv error: recv got EOF from 127.0.0.1:44854 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
W20260809 16:24:55.359833 23755 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:24:55.359044 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44856 (local address 127.21.51.189:44933)
0809 16:24:55.359081 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0809 16:24:55.359083 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:24:55.359304 (+ 221us) server_negotiation.cc:407] Connection header received
0809 16:24:55.359400 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:24:55.359401 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:24:55.359412 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:24:55.359442 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:24:55.359777 (+ 335us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44856: BlockingRecv error: recv got EOF from 127.0.0.1:44856 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260809 16:24:55.378381 23755 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:24:55.377397 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44872 (local address 127.21.51.189:44933)
0809 16:24:55.377443 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0809 16:24:55.377445 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:24:55.377722 (+ 277us) server_negotiation.cc:407] Connection header received
0809 16:24:55.377832 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:24:55.377833 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:24:55.377845 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:24:55.377875 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:24:55.378318 (+ 443us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44872: BlockingRecv error: recv got EOF from 127.0.0.1:44872 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
W20260809 16:24:55.413017 23755 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:24:55.411977 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44888 (local address 127.21.51.189:44933)
0809 16:24:55.412026 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0809 16:24:55.412028 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:24:55.412317 (+ 289us) server_negotiation.cc:407] Connection header received
0809 16:24:55.412432 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:24:55.412433 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:24:55.412446 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:24:55.412478 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:24:55.412955 (+ 477us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44888: BlockingRecv error: recv got EOF from 127.0.0.1:44888 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
W20260809 16:24:55.479346 23755 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:24:55.478558 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44900 (local address 127.21.51.189:44933)
0809 16:24:55.478673 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0809 16:24:55.478675 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:24:55.478702 (+ 27us) server_negotiation.cc:407] Connection header received
0809 16:24:55.478787 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:24:55.478788 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:24:55.478799 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:24:55.478855 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:24:55.479271 (+ 416us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44900: BlockingRecv error: recv got EOF from 127.0.0.1:44900 (error 108)
Metrics: {"server-negotiator.queue_time_us":84}
W20260809 16:24:55.609323 23755 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:24:55.608610 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44914 (local address 127.21.51.189:44933)
0809 16:24:55.608724 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0809 16:24:55.608726 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:24:55.608741 (+ 15us) server_negotiation.cc:407] Connection header received
0809 16:24:55.608785 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:24:55.608786 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:24:55.608797 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:24:55.608855 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:24:55.609216 (+ 361us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44914: BlockingRecv error: recv got EOF from 127.0.0.1:44914 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
W20260809 16:24:55.867257 23755 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:24:55.866541 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44918 (local address 127.21.51.189:44933)
0809 16:24:55.866597 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0809 16:24:55.866604 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 16:24:55.866627 (+ 23us) server_negotiation.cc:407] Connection header received
0809 16:24:55.866701 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:24:55.866702 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:24:55.866713 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:24:55.866755 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:24:55.867164 (+ 409us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44918: BlockingRecv error: recv got EOF from 127.0.0.1:44918 (error 108)
Metrics: {"server-negotiator.queue_time_us":27}
I20260809 16:24:56.219905 23933 catalog_manager.cc:1259] Loaded cluster ID: 00d596e2e7b84385830a86b3048d2204
I20260809 16:24:56.219985 23933 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 9d61eb883bce4c6eb03609d81a8600f8: loading cluster ID for follower catalog manager: success
I20260809 16:24:56.220831 23933 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 9d61eb883bce4c6eb03609d81a8600f8: acquiring CA information for follower catalog manager: success
I20260809 16:24:56.221158 23933 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 9d61eb883bce4c6eb03609d81a8600f8: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 16:24:56.277643 23971 catalog_manager.cc:1259] Loaded cluster ID: 00d596e2e7b84385830a86b3048d2204
I20260809 16:24:56.277720 23971 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 4fc0017b5efd45aebd25c4c6a6ac994b: loading cluster ID for follower catalog manager: success
I20260809 16:24:56.278558 23971 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 4fc0017b5efd45aebd25c4c6a6ac994b: acquiring CA information for follower catalog manager: success
I20260809 16:24:56.278870 23971 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 4fc0017b5efd45aebd25c4c6a6ac994b: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 16:24:56.383247 24084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:56.380514 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44928 (local address 127.21.51.189:44933)
0809 16:24:56.380647 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0809 16:24:56.380650 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 16:24:56.380779 (+ 129us) server_negotiation.cc:407] Connection header received
0809 16:24:56.380864 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:24:56.380865 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:24:56.380957 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:24:56.381051 (+ 94us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 16:24:56.381228 (+ 177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:24:56.381721 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:24:56.382472 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:24:56.382711 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:24:56.382780 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:24:56.382920 (+ 140us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 16:24:56.383037 (+ 117us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 16:24:56.383105 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0809 16:24:56.383138 (+ 33us) server_negotiation.cc:299] Negotiation successful
0809 16:24:56.383159 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":103,"thread_start_us":78,"threads_started":1}
I20260809 16:24:56.388221 24094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:56.385260 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43030 (local address 127.21.51.188:40535)
0809 16:24:56.385387 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0809 16:24:56.385390 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 16:24:56.385400 (+ 10us) server_negotiation.cc:407] Connection header received
0809 16:24:56.385449 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:24:56.385450 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:24:56.385504 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:24:56.385599 (+ 95us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 16:24:56.385784 (+ 185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:24:56.386317 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:24:56.387166 (+ 849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:24:56.387402 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:24:56.387486 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:24:56.387656 (+ 170us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 16:24:56.387938 (+ 282us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 16:24:56.388052 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0809 16:24:56.388106 (+ 54us) server_negotiation.cc:299] Negotiation successful
0809 16:24:56.388128 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":102,"thread_start_us":49,"threads_started":1}
I20260809 16:24:56.392808 24084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:56.390034 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44940 (local address 127.21.51.189:44933)
0809 16:24:56.390094 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0809 16:24:56.390096 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:24:56.390105 (+ 9us) server_negotiation.cc:407] Connection header received
0809 16:24:56.390131 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:24:56.390132 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:24:56.390141 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:24:56.390185 (+ 44us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 16:24:56.390316 (+ 131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:24:56.390956 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:24:56.391943 (+ 987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:24:56.392263 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:24:56.392299 (+ 36us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:24:56.392489 (+ 190us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 16:24:56.392527 (+ 38us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 16:24:56.392591 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0809 16:24:56.392672 (+ 81us) server_negotiation.cc:299] Negotiation successful
0809 16:24:56.392714 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260809 16:24:56.393131 24094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:56.390118 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43038 (local address 127.21.51.188:40535)
0809 16:24:56.390233 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0809 16:24:56.390235 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:24:56.390243 (+ 8us) server_negotiation.cc:407] Connection header received
0809 16:24:56.390263 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:24:56.390264 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:24:56.390273 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:24:56.390324 (+ 51us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 16:24:56.390510 (+ 186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:24:56.391218 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:24:56.392228 (+ 1010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:24:56.392516 (+ 288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:24:56.392584 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:24:56.392860 (+ 276us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 16:24:56.392887 (+ 27us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 16:24:56.392912 (+ 25us) server_negotiation.cc:1035] Waiting for connection context
0809 16:24:56.393037 (+ 125us) server_negotiation.cc:299] Negotiation successful
0809 16:24:56.393059 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95}
I20260809 16:24:56.393339 24104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:56.389914 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48534 (local address 127.21.51.190:45837)
0809 16:24:56.390323 (+ 409us) server_negotiation.cc:206] Beginning negotiation
0809 16:24:56.390325 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:24:56.390335 (+ 10us) server_negotiation.cc:407] Connection header received
0809 16:24:56.390473 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:24:56.390474 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:24:56.390522 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:24:56.390602 (+ 80us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 16:24:56.390748 (+ 146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:24:56.391455 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:24:56.392245 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:24:56.392554 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:24:56.392610 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:24:56.392861 (+ 251us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 16:24:56.392990 (+ 129us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 16:24:56.393053 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0809 16:24:56.393218 (+ 165us) server_negotiation.cc:299] Negotiation successful
0809 16:24:56.393248 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":385,"thread_start_us":62,"threads_started":1}
I20260809 16:24:56.394908 21710 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrNqUCn/build/release/bin/kudu with pid 23527
I20260809 16:24:56.407488 21710 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrNqUCn/build/release/bin/kudu with pid 23657
I20260809 16:24:56.415757 21710 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrNqUCn/build/release/bin/kudu with pid 23761
2026-08-09T16:24:56Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (3030 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786292417722232-21710-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 16:24:56 dist-test-slave-s11t krb5kdc[24112](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 16:24:56 dist-test-slave-s11t krb5kdc[24112](info): set up 2 sockets
Aug 09 16:24:56 dist-test-slave-s11t krb5kdc[24112](info): commencing operation
krb5kdc: starting...
W20260809 16:24:57.654331 21710 mini_kdc.cc:121] Time spent starting KDC: real 1.215s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786292417722232-21710-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786292417722232-21710-0/krb5kdc/test-user.keytab.
Aug 09 16:24:57 dist-test-slave-s11t krb5kdc[24112](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292697, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T16:24:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T16:24:57Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.51.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.51.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.51.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.190.keytab.
Entry for principal kudu/127.21.51.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.190.keytab.
I20260809 16:24:57.775614 21710 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrNqUCn/build/release/bin/kudu
/tmp/dist-test-taskrNqUCn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786292417722232-21710-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786292417722232-21710-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786292417722232-21710-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786292417722232-21710-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.51.190:45669
--webserver_interface=127.21.51.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.190.keytab
--principal=kudu/127.21.51.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.51.148:43965
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.51.190:45669,127.21.51.189:41049,127.21.51.188:45351
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786292417722232-21710-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786292417722232-21710-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786292417722232-21710-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:24:57.858000 24134 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260809 16:24:57.858263 24134 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:24:57.858299 24134 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:24:57.858340 24134 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:24:57.859450 24134 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 16:24:57.859520 24134 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:24:57.859543 24134 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:24:57.859565 24134 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 16:24:57.859596 24134 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 16:24:57.861093 24134 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.51.148:43965
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786292417722232-21710-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786292417722232-21710-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.51.190:45669,127.21.51.189:41049,127.21.51.188:45351
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.190.keytab
--principal=kudu/127.21.51.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.51.190:45669
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786292417722232-21710-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.51.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24134
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786292417722232-21710-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:24:57.861364 24134 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:24:57.861610 24134 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:24:57.866627 24140 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:24:57.866631 24141 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:24:57.866691 24143 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:24:57.866930 24134 server_base.cc:1034] running on GCE node
I20260809 16:24:57.867143 24134 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:24:57.867415 24134 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:24:57.868574 24134 hybrid_clock.cc:648] HybridClock initialized: now 1786292697868564 us; error 31 us; skew 500 ppm
Aug 09 16:24:57 dist-test-slave-s11t krb5kdc[24112](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292697, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.51.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:24:57.875424 24134 init.cc:377] Logged in from keytab as kudu/127.21.51.190@KRBTEST.COM (short username kudu)
I20260809 16:24:57.875885 24134 webserver.cc:460] Webserver started at http://127.21.51.190:46289/ using document root <none> and password file <none>
I20260809 16:24:57.876096 24134 fs_manager.cc:359] Metadata directory not provided
I20260809 16:24:57.876137 24134 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:24:57.876225 24134 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:24:57.877303 24134 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786292417722232-21710-0/minicluster-data/master-0/data/instance:
uuid: "6d7cb2032f954c669241ffa084b9bda0"
format_stamp: "Formatted at 2026-08-09 16:24:57 on dist-test-slave-s11t"
server_key: "ab75dece15c2335ac8b74d56dd57c950"
server_key_iv: "6440d242963cd964b7a321ce7294b4a7"
server_key_version: "encryptionkey@0"
I20260809 16:24:57.877597 24134 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786292417722232-21710-0/minicluster-data/master-0/wal/instance:
uuid: "6d7cb2032f954c669241ffa084b9bda0"
format_stamp: "Formatted at 2026-08-09 16:24:57 on dist-test-slave-s11t"
server_key: "ab75dece15c2335ac8b74d56dd57c950"
server_key_iv: "6440d242963cd964b7a321ce7294b4a7"
server_key_version: "encryptionkey@0"
I20260809 16:24:57.878845 24134 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 16:24:57.879570 24151 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:24:57.879747 24134 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 16:24:57.879856 24134 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786292417722232-21710-0/minicluster-data/master-0/data,/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786292417722232-21710-0/minicluster-data/master-0/wal
uuid: "6d7cb2032f954c669241ffa084b9bda0"
format_stamp: "Formatted at 2026-08-09 16:24:57 on dist-test-slave-s11t"
server_key: "ab75dece15c2335ac8b74d56dd57c950"
server_key_iv: "6440d242963cd964b7a321ce7294b4a7"
server_key_version: "encryptionkey@0"
I20260809 16:24:57.879941 24134 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786292417722232-21710-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786292417722232-21710-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786292417722232-21710-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:24:57.891214 24134 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:24:57.893261 24134 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:24:57.893419 24134 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:24:57.897671 24134 rpc_server.cc:307] RPC server started. Bound to: 127.21.51.190:45669
I20260809 16:24:57.897706 24223 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.51.190:45669 every 8 connection(s)
I20260809 16:24:57.898085 24134 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786292417722232-21710-0/minicluster-data/master-0/data/info.pb
I20260809 16:24:57.899060 24225 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:24:57.900806 21710 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrNqUCn/build/release/bin/kudu as pid 24134
I20260809 16:24:57.900574 24225 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.51.190" port: 45669 } has no permanent_uuid. Determining permanent_uuid...
I20260809 16:24:57.900979 21710 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786292417722232-21710-0/minicluster-data/master-0/wal/instance
I20260809 16:24:57.901178 21710 external_mini_cluster.cc:1428] Setting key 815ff4e43fe81970e29d677cf77de37a
Aug 09 16:24:57 dist-test-slave-s11t krb5kdc[24112](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292697, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.51.190@KRBTEST.COM for kudu/127.21.51.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.21.51.189@KRBTEST.COM; defaulting to no policy
I20260809 16:24:57.912114 24227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:57.901366 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:45669 (local address 127.0.0.1:55004)
0809 16:24:57.901592 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0809 16:24:57.901636 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 16:24:57.902338 (+ 702us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:24:57.902635 (+ 297us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:24:57.902637 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:24:57.903023 (+ 386us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:24:57.903263 (+ 240us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:24:57.903269 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:24:57.904801 (+ 1532us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:24:57.904803 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:24:57.905979 (+ 1176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:24:57.905983 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:24:57.906487 (+ 504us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:24:57.906488 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:24:57.906577 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:24:57.907082 (+ 505us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:24:57.907094 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:24:57.908766 (+ 1672us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:24:57.910592 (+ 1826us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:24:57.910599 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:24:57.910601 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:24:57.910799 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:24:57.911065 (+ 266us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:24:57.911066 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:24:57.911066 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:24:57.911153 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:24:57.911516 (+ 363us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:24:57.911521 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:24:57.911584 (+ 63us) client_negotiation.cc:769] Sending connection context
0809 16:24:57.911683 (+ 99us) client_negotiation.cc:240] Negotiation successful
0809 16:24:57.911921 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":163,"mutex_wait_us":99,"thread_start_us":100,"threads_started":1}
I20260809 16:24:57.912117 24231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:57.901425 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55004 (local address 127.21.51.190:45669)
0809 16:24:57.901597 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0809 16:24:57.901607 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 16:24:57.901735 (+ 128us) server_negotiation.cc:407] Connection header received
0809 16:24:57.902498 (+ 763us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:24:57.902500 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:24:57.902517 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:24:57.902595 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:24:57.903368 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:24:57.904670 (+ 1302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:24:57.906061 (+ 1391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:24:57.906367 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:24:57.906455 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:24:57.908859 (+ 2404us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:24:57.908866 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:24:57.908867 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:24:57.908876 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:24:57.910506 (+ 1630us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:24:57.910899 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:24:57.910902 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:24:57.910903 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:24:57.910987 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:24:57.911217 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:24:57.911218 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:24:57.911219 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:24:57.911433 (+ 214us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:24:57.911524 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0809 16:24:57.911731 (+ 207us) server_negotiation.cc:299] Negotiation successful
0809 16:24:57.911866 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":137,"thread_start_us":74,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.51.189@KRBTEST.COM" created.
I20260809 16:24:57.912992 24225 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.51.189" port: 41049 } has no permanent_uuid. Determining permanent_uuid...
I20260809 16:24:57.913378 24227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:57.913234 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.189:41049 (local address 127.0.0.1:55442)
0809 16:24:57.913290 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0809 16:24:57.913343 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.51.189:41049: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260809 16:24:57.913552 24155 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.21.51.189:41049: connect: Connection refused (error 111)
I20260809 16:24:57.914016 24227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:57.913908 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.189:41049 (local address 127.0.0.1:55454)
0809 16:24:57.913954 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0809 16:24:57.913984 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.51.189:41049: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260809 16:24:57.914132 24225 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.51.189:41049: Network error: Client connection negotiation failed: client connection to 127.21.51.189:41049: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.51.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.189.keytab.
Entry for principal kudu/127.21.51.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.189.keytab.
I20260809 16:24:57.927599 21710 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrNqUCn/build/release/bin/kudu
/tmp/dist-test-taskrNqUCn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786292417722232-21710-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786292417722232-21710-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786292417722232-21710-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786292417722232-21710-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.51.189:41049
--webserver_interface=127.21.51.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.189.keytab
--principal=kudu/127.21.51.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.51.148:43965
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.51.190:45669,127.21.51.189:41049,127.21.51.188:45351
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786292417722232-21710-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786292417722232-21710-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786292417722232-21710-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 16:24:57.963359 24225 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.51.189" port: 41049 } attempt: 1
I20260809 16:24:57.963855 24227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:57.963718 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.189:41049 (local address 127.0.0.1:55466)
0809 16:24:57.963786 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0809 16:24:57.963825 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.51.189:41049: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260809 16:24:57.964238 24227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:57.964136 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.189:41049 (local address 127.0.0.1:55472)
0809 16:24:57.964185 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0809 16:24:57.964218 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.51.189:41049: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260809 16:24:57.964334 24225 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.51.189:41049: Network error: Client connection negotiation failed: client connection to 127.21.51.189:41049: connect: Connection refused (error 111)
W20260809 16:24:58.012560 24238 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260809 16:24:58.012856 24238 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:24:58.012908 24238 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:24:58.012949 24238 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:24:58.014076 24238 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 16:24:58.014145 24238 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:24:58.014171 24238 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:24:58.014199 24238 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 16:24:58.014225 24238 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 16:24:58.015714 24238 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.51.148:43965
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786292417722232-21710-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786292417722232-21710-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.21.51.190:45669,127.21.51.189:41049,127.21.51.188:45351
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.189.keytab
--principal=kudu/127.21.51.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.51.189:41049
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786292417722232-21710-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.21.51.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24238
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786292417722232-21710-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:24:58.016042 24238 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:24:58.016285 24238 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:24:58.021471 24245 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:24:58.021569 24244 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:24:58.021453 24250 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:24:58.021685 24238 server_base.cc:1034] running on GCE node
I20260809 16:24:58.021926 24238 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:24:58.022240 24238 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:24:58.023425 24238 hybrid_clock.cc:648] HybridClock initialized: now 1786292698023404 us; error 56 us; skew 500 ppm
Aug 09 16:24:58 dist-test-slave-s11t krb5kdc[24112](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292698, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.51.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:24:58.026140 24238 init.cc:377] Logged in from keytab as kudu/127.21.51.189@KRBTEST.COM (short username kudu)
I20260809 16:24:58.026623 24238 webserver.cc:460] Webserver started at http://127.21.51.189:34193/ using document root <none> and password file <none>
I20260809 16:24:58.026861 24238 fs_manager.cc:359] Metadata directory not provided
I20260809 16:24:58.026937 24238 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:24:58.027086 24238 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:24:58.028137 24238 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786292417722232-21710-0/minicluster-data/master-1/data/instance:
uuid: "bb957a45319c4018a8fc6cf78f8d668e"
format_stamp: "Formatted at 2026-08-09 16:24:58 on dist-test-slave-s11t"
server_key: "0cc6e31fa118032d772805923c68fe9a"
server_key_iv: "1ffe3e540ea6c516f0ab780b24e374f7"
server_key_version: "encryptionkey@0"
I20260809 16:24:58.028496 24238 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786292417722232-21710-0/minicluster-data/master-1/wal/instance:
uuid: "bb957a45319c4018a8fc6cf78f8d668e"
format_stamp: "Formatted at 2026-08-09 16:24:58 on dist-test-slave-s11t"
server_key: "0cc6e31fa118032d772805923c68fe9a"
server_key_iv: "1ffe3e540ea6c516f0ab780b24e374f7"
server_key_version: "encryptionkey@0"
I20260809 16:24:58.029915 24238 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260809 16:24:58.030656 24264 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:24:58.030827 24238 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 16:24:58.030951 24238 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786292417722232-21710-0/minicluster-data/master-1/data,/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786292417722232-21710-0/minicluster-data/master-1/wal
uuid: "bb957a45319c4018a8fc6cf78f8d668e"
format_stamp: "Formatted at 2026-08-09 16:24:58 on dist-test-slave-s11t"
server_key: "0cc6e31fa118032d772805923c68fe9a"
server_key_iv: "1ffe3e540ea6c516f0ab780b24e374f7"
server_key_version: "encryptionkey@0"
I20260809 16:24:58.031035 24238 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786292417722232-21710-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786292417722232-21710-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786292417722232-21710-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:24:58.032459 24225 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.51.189" port: 41049 } attempt: 2
I20260809 16:24:58.032863 24227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:58.032734 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.189:41049 (local address 127.0.0.1:55482)
0809 16:24:58.032798 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0809 16:24:58.032836 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.51.189:41049: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260809 16:24:58.033170 24227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:58.033073 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.189:41049 (local address 127.0.0.1:55494)
0809 16:24:58.033118 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0809 16:24:58.033142 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.51.189:41049: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260809 16:24:58.033254 24225 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.51.189:41049: Network error: Client connection negotiation failed: client connection to 127.21.51.189:41049: connect: Connection refused (error 111)
I20260809 16:24:58.039930 24238 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:24:58.042291 24238 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:24:58.042464 24238 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:24:58.046614 24238 rpc_server.cc:307] RPC server started. Bound to: 127.21.51.189:41049
I20260809 16:24:58.046646 24353 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.51.189:41049 every 8 connection(s)
I20260809 16:24:58.047003 24238 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786292417722232-21710-0/minicluster-data/master-1/data/info.pb
I20260809 16:24:58.047987 24354 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:24:58.049642 24354 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.51.190" port: 45669 } has no permanent_uuid. Determining permanent_uuid...
I20260809 16:24:58.053126 21710 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrNqUCn/build/release/bin/kudu as pid 24238
I20260809 16:24:58.053243 21710 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786292417722232-21710-0/minicluster-data/master-1/wal/instance
I20260809 16:24:58.053457 21710 external_mini_cluster.cc:1428] Setting key 26ecc9358b3229075d022fb81642d4b0
Aug 09 16:24:58 dist-test-slave-s11t krb5kdc[24112](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292698, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.51.189@KRBTEST.COM for kudu/127.21.51.190@KRBTEST.COM
I20260809 16:24:58.058959 24231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:58.050330 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55016 (local address 127.21.51.190:45669)
0809 16:24:58.050390 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0809 16:24:58.050409 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0809 16:24:58.050613 (+ 204us) server_negotiation.cc:407] Connection header received
0809 16:24:58.051136 (+ 523us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:24:58.051138 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:24:58.051155 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:24:58.051203 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:24:58.051841 (+ 638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:24:58.052337 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:24:58.053471 (+ 1134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:24:58.053723 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:24:58.053812 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:24:58.055971 (+ 2159us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:24:58.055972 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:24:58.055973 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:24:58.055980 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:24:58.057536 (+ 1556us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:24:58.057983 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:24:58.057985 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:24:58.057985 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:24:58.058050 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:24:58.058312 (+ 262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:24:58.058313 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:24:58.058314 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:24:58.058431 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:24:58.058508 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0809 16:24:58.058779 (+ 271us) server_negotiation.cc:299] Negotiation successful
0809 16:24:58.058865 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260809 16:24:58.059168 24356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:58.050286 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:45669 (local address 127.0.0.1:55016)
0809 16:24:58.050486 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0809 16:24:58.050524 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0809 16:24:58.051025 (+ 501us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:24:58.051286 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:24:58.051290 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:24:58.051553 (+ 263us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:24:58.051740 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:24:58.051745 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:24:58.052444 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:24:58.052445 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:24:58.053384 (+ 939us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:24:58.053387 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:24:58.053843 (+ 456us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:24:58.053845 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:24:58.053977 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:24:58.054516 (+ 539us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:24:58.054524 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:24:58.055869 (+ 1345us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:24:58.057645 (+ 1776us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:24:58.057653 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:24:58.057655 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:24:58.057893 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:24:58.058121 (+ 228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:24:58.058121 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:24:58.058122 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:24:58.058229 (+ 107us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:24:58.058500 (+ 271us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:24:58.058507 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:24:58.058645 (+ 138us) client_negotiation.cc:769] Sending connection context
0809 16:24:58.058757 (+ 112us) client_negotiation.cc:240] Negotiation successful
0809 16:24:58.058921 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":94,"threads_started":1}
I20260809 16:24:58.059653 24354 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.51.189" port: 41049 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.21.51.188@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.51.188@KRBTEST.COM" created.
Aug 09 16:24:58 dist-test-slave-s11t krb5kdc[24112](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292698, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.51.189@KRBTEST.COM for kudu/127.21.51.189@KRBTEST.COM
I20260809 16:24:58.068111 24356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:58.059938 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.189:41049 (local address 127.0.0.1:55508)
0809 16:24:58.059979 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0809 16:24:58.059989 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 16:24:58.060026 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:24:58.060788 (+ 762us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:24:58.060789 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:24:58.060930 (+ 141us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:24:58.060978 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:24:58.060979 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:24:58.062333 (+ 1354us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:24:58.062334 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:24:58.062857 (+ 523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:24:58.062891 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:24:58.063482 (+ 591us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:24:58.063483 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:24:58.063549 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:24:58.063872 (+ 323us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:24:58.063875 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:24:58.065312 (+ 1437us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:24:58.067030 (+ 1718us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:24:58.067031 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:24:58.067032 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:24:58.067196 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:24:58.067413 (+ 217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:24:58.067414 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:24:58.067414 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:24:58.067469 (+ 55us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:24:58.067763 (+ 294us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:24:58.067764 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:24:58.067811 (+ 47us) client_negotiation.cc:769] Sending connection context
0809 16:24:58.067897 (+ 86us) client_negotiation.cc:240] Negotiation successful
0809 16:24:58.067998 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":27}
I20260809 16:24:58.068199 24358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:58.059988 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55508 (local address 127.21.51.189:41049)
0809 16:24:58.060601 (+ 613us) server_negotiation.cc:206] Beginning negotiation
0809 16:24:58.060606 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 16:24:58.060633 (+ 27us) server_negotiation.cc:407] Connection header received
0809 16:24:58.060657 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:24:58.060659 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:24:58.060672 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:24:58.060779 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:24:58.061063 (+ 284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:24:58.062220 (+ 1157us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:24:58.063100 (+ 880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:24:58.063394 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:24:58.063474 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:24:58.065398 (+ 1924us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:24:58.065406 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:24:58.065406 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:24:58.065413 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:24:58.066926 (+ 1513us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:24:58.067279 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:24:58.067282 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:24:58.067282 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:24:58.067346 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:24:58.067541 (+ 195us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:24:58.067542 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:24:58.067543 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:24:58.067681 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:24:58.067770 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0809 16:24:58.067931 (+ 161us) server_negotiation.cc:299] Negotiation successful
0809 16:24:58.068070 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":56,"server-negotiator.queue_time_us":577,"thread_start_us":71,"threads_started":1}
I20260809 16:24:58.068759 24354 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.51.188" port: 45351 } has no permanent_uuid. Determining permanent_uuid...
I20260809 16:24:58.069113 24356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:58.068996 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.188:45351 (local address 127.0.0.1:49234)
0809 16:24:58.069044 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0809 16:24:58.069089 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.51.188:45351: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260809 16:24:58.069247 24274 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.21.51.188:45351: connect: Connection refused (error 111)
I20260809 16:24:58.069705 24356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:58.069611 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.188:45351 (local address 127.0.0.1:49248)
0809 16:24:58.069659 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0809 16:24:58.069683 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.51.188:45351: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260809 16:24:58.069818 24354 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.51.188:45351: Network error: Client connection negotiation failed: client connection to 127.21.51.188:45351: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.51.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.188.keytab.
Entry for principal kudu/127.21.51.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.188.keytab.
I20260809 16:24:58.079304 21710 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrNqUCn/build/release/bin/kudu
/tmp/dist-test-taskrNqUCn/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786292417722232-21710-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786292417722232-21710-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786292417722232-21710-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786292417722232-21710-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.51.188:45351
--webserver_interface=127.21.51.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.188.keytab
--principal=kudu/127.21.51.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.51.148:43965
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.51.190:45669,127.21.51.189:41049,127.21.51.188:45351
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786292417722232-21710-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786292417722232-21710-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786292417722232-21710-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 16:24:58.119026 24354 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.51.188" port: 45351 } attempt: 1
I20260809 16:24:58.119499 24356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:58.119356 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.188:45351 (local address 127.0.0.1:49254)
0809 16:24:58.119428 (+ 72us) negotiation.cc:107] Waiting for socket to connect
0809 16:24:58.119466 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.51.188:45351: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260809 16:24:58.119850 24356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:58.119754 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.188:45351 (local address 127.0.0.1:49270)
0809 16:24:58.119799 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0809 16:24:58.119824 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.51.188:45351: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260809 16:24:58.119958 24354 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.51.188:45351: Network error: Client connection negotiation failed: client connection to 127.21.51.188:45351: connect: Connection refused (error 111)
I20260809 16:24:58.124380 24225 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.51.189" port: 41049 } attempt: 3
Aug 09 16:24:58 dist-test-slave-s11t krb5kdc[24112](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292697, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.51.190@KRBTEST.COM for kudu/127.21.51.189@KRBTEST.COM
I20260809 16:24:58.131455 24358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:58.124738 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55524 (local address 127.21.51.189:41049)
0809 16:24:58.124800 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0809 16:24:58.124802 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:24:58.124812 (+ 10us) server_negotiation.cc:407] Connection header received
0809 16:24:58.124876 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:24:58.124877 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:24:58.124888 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:24:58.124963 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:24:58.125287 (+ 324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:24:58.125792 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:24:58.126514 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:24:58.126740 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:24:58.126829 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:24:58.128959 (+ 2130us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:24:58.128960 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:24:58.128961 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:24:58.128967 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:24:58.130340 (+ 1373us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:24:58.130742 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:24:58.130743 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:24:58.130744 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:24:58.130801 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:24:58.131008 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:24:58.131008 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:24:58.131009 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:24:58.131114 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:24:58.131165 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0809 16:24:58.131325 (+ 160us) server_negotiation.cc:299] Negotiation successful
0809 16:24:58.131386 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260809 16:24:58.131599 24227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:58.124660 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.189:41049 (local address 127.0.0.1:55524)
0809 16:24:58.124725 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0809 16:24:58.124736 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 16:24:58.124808 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:24:58.124969 (+ 161us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:24:58.124970 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:24:58.125138 (+ 168us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:24:58.125204 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:24:58.125206 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:24:58.125879 (+ 673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:24:58.125881 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:24:58.126427 (+ 546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:24:58.126429 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:24:58.126826 (+ 397us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:24:58.126827 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:24:58.126914 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:24:58.127372 (+ 458us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:24:58.127377 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:24:58.128840 (+ 1463us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:24:58.130425 (+ 1585us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:24:58.130426 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:24:58.130427 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:24:58.130650 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:24:58.130872 (+ 222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:24:58.130873 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:24:58.130873 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:24:58.130944 (+ 71us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:24:58.131177 (+ 233us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:24:58.131178 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:24:58.131232 (+ 54us) client_negotiation.cc:769] Sending connection context
0809 16:24:58.131309 (+ 77us) client_negotiation.cc:240] Negotiation successful
0809 16:24:58.131417 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":45}
I20260809 16:24:58.132030 24225 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.51.188" port: 45351 } has no permanent_uuid. Determining permanent_uuid...
I20260809 16:24:58.132381 24227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:58.132264 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.188:45351 (local address 127.0.0.1:49286)
0809 16:24:58.132314 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0809 16:24:58.132356 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.51.188:45351: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260809 16:24:58.132735 24227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:58.132617 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.188:45351 (local address 127.0.0.1:49294)
0809 16:24:58.132662 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0809 16:24:58.132715 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.51.188:45351: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260809 16:24:58.132833 24225 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.51.188:45351: Network error: Client connection negotiation failed: client connection to 127.21.51.188:45351: connect: Connection refused (error 111)
W20260809 16:24:58.163169 24362 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260809 16:24:58.163378 24362 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:24:58.163395 24362 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:24:58.163414 24362 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260809 16:24:58.163992 24225 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.51.188" port: 45351 } attempt: 1
I20260809 16:24:58.164463 24227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:58.164306 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.188:45351 (local address 127.0.0.1:49306)
0809 16:24:58.164376 (+ 70us) negotiation.cc:107] Waiting for socket to connect
0809 16:24:58.164431 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.51.188:45351: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260809 16:24:58.164865 24362 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
I20260809 16:24:58.164872 24227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:58.164786 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.188:45351 (local address 127.0.0.1:49312)
0809 16:24:58.164827 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0809 16:24:58.164854 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.51.188:45351: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
W20260809 16:24:58.164908 24362 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:24:58.164925 24362 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:24:58.164959 24362 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 16:24:58.164980 24362 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260809 16:24:58.164994 24225 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.51.188:45351: Network error: Client connection negotiation failed: client connection to 127.21.51.188:45351: connect: Connection refused (error 111)
I20260809 16:24:58.166495 24362 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.51.148:43965
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786292417722232-21710-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786292417722232-21710-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.21.51.190:45669,127.21.51.189:41049,127.21.51.188:45351
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786292417722232-21710-0/krb5kdc/kudu_127.21.51.188.keytab
--principal=kudu/127.21.51.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.51.188:45351
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786292417722232-21710-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.21.51.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24362
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786292417722232-21710-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:59:38 UTC on ccycloud.quasar-muncrq.root.comops.site
build id 4576545
I20260809 16:24:58.166746 24362 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:24:58.167001 24362 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:24:58.171778 24378 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:24:58.171789 24370 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:24:58.171883 24362 server_base.cc:1034] running on GCE node
W20260809 16:24:58.172060 24369 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:24:58.172287 24362 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:24:58.172598 24362 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:24:58.173763 24362 hybrid_clock.cc:648] HybridClock initialized: now 1786292698173769 us; error 49 us; skew 500 ppm
Aug 09 16:24:58 dist-test-slave-s11t krb5kdc[24112](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292698, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.51.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:24:58.176160 24362 init.cc:377] Logged in from keytab as kudu/127.21.51.188@KRBTEST.COM (short username kudu)
I20260809 16:24:58.176589 24362 webserver.cc:460] Webserver started at http://127.21.51.188:41981/ using document root <none> and password file <none>
I20260809 16:24:58.176865 24362 fs_manager.cc:359] Metadata directory not provided
I20260809 16:24:58.176944 24362 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:24:58.177103 24362 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:24:58.178140 24362 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786292417722232-21710-0/minicluster-data/master-2/data/instance:
uuid: "219add3b418143088dcc3c25683ffa23"
format_stamp: "Formatted at 2026-08-09 16:24:58 on dist-test-slave-s11t"
server_key: "66bea953a444a014f43eeef76e24d53a"
server_key_iv: "9db3579063da6a2d6b55f1f98f280a05"
server_key_version: "encryptionkey@0"
I20260809 16:24:58.178556 24362 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786292417722232-21710-0/minicluster-data/master-2/wal/instance:
uuid: "219add3b418143088dcc3c25683ffa23"
format_stamp: "Formatted at 2026-08-09 16:24:58 on dist-test-slave-s11t"
server_key: "66bea953a444a014f43eeef76e24d53a"
server_key_iv: "9db3579063da6a2d6b55f1f98f280a05"
server_key_version: "encryptionkey@0"
I20260809 16:24:58.180456 24362 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20260809 16:24:58.181548 24390 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:24:58.181749 24362 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:24:58.181860 24362 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786292417722232-21710-0/minicluster-data/master-2/data,/tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786292417722232-21710-0/minicluster-data/master-2/wal
uuid: "219add3b418143088dcc3c25683ffa23"
format_stamp: "Formatted at 2026-08-09 16:24:58 on dist-test-slave-s11t"
server_key: "66bea953a444a014f43eeef76e24d53a"
server_key_iv: "9db3579063da6a2d6b55f1f98f280a05"
server_key_version: "encryptionkey@0"
I20260809 16:24:58.181954 24362 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786292417722232-21710-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786292417722232-21710-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786292417722232-21710-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:24:58.188100 24354 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.51.188" port: 45351 } attempt: 2
I20260809 16:24:58.188534 24356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:58.188408 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.188:45351 (local address 127.0.0.1:49326)
0809 16:24:58.188463 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0809 16:24:58.188502 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.51.188:45351: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260809 16:24:58.188925 24356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:58.188828 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.188:45351 (local address 127.0.0.1:49330)
0809 16:24:58.188872 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0809 16:24:58.188900 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.51.188:45351: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260809 16:24:58.189041 24354 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.51.188:45351: Network error: Client connection negotiation failed: client connection to 127.21.51.188:45351: connect: Connection refused (error 111)
I20260809 16:24:58.192821 24362 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:24:58.194618 24362 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:24:58.194772 24362 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:24:58.198694 24362 rpc_server.cc:307] RPC server started. Bound to: 127.21.51.188:45351
I20260809 16:24:58.198797 24471 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.51.188:45351 every 8 connection(s)
I20260809 16:24:58.199122 24362 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786292417722232-21710-0/minicluster-data/master-2/data/info.pb
I20260809 16:24:58.200018 24472 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:24:58.201576 24472 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.51.190" port: 45669 } has no permanent_uuid. Determining permanent_uuid...
I20260809 16:24:58.204399 21710 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrNqUCn/build/release/bin/kudu as pid 24362
I20260809 16:24:58.204525 21710 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrNqUCn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786292417722232-21710-0/minicluster-data/master-2/wal/instance
I20260809 16:24:58.204782 21710 external_mini_cluster.cc:1428] Setting key 4c9483798e6e8a3ede14c4dd440eff10
Aug 09 16:24:58 dist-test-slave-s11t krb5kdc[24112](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292698, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.51.188@KRBTEST.COM for kudu/127.21.51.190@KRBTEST.COM
Aug 09 16:24:58 dist-test-slave-s11t krb5kdc[24112](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292697, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.51.188@KRBTEST.COM
I20260809 16:24:58.211344 24231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:58.202204 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55028 (local address 127.21.51.190:45669)
0809 16:24:58.202267 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0809 16:24:58.202273 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 16:24:58.202493 (+ 220us) server_negotiation.cc:407] Connection header received
0809 16:24:58.203015 (+ 522us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:24:58.203017 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:24:58.203030 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:24:58.203077 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:24:58.203704 (+ 627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:24:58.204195 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:24:58.205559 (+ 1364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:24:58.205750 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:24:58.205825 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:24:58.208249 (+ 2424us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:24:58.208250 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:24:58.208251 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:24:58.208257 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:24:58.209932 (+ 1675us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:24:58.210321 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:24:58.210322 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:24:58.210323 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:24:58.210385 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:24:58.210629 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:24:58.210630 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:24:58.210631 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:24:58.210768 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:24:58.210844 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0809 16:24:58.211137 (+ 293us) server_negotiation.cc:299] Negotiation successful
0809 16:24:58.211223 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260809 16:24:58.211413 24474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:58.202159 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.190:45669 (local address 127.0.0.1:55028)
0809 16:24:58.202362 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0809 16:24:58.202411 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0809 16:24:58.202896 (+ 485us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:24:58.203158 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:24:58.203161 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:24:58.203408 (+ 247us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:24:58.203594 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:24:58.203598 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:24:58.204293 (+ 695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:24:58.204294 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:24:58.205470 (+ 1176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:24:58.205473 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:24:58.205865 (+ 392us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:24:58.205867 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:24:58.205985 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:24:58.206455 (+ 470us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:24:58.206464 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:24:58.208143 (+ 1679us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:24:58.210016 (+ 1873us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:24:58.210022 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:24:58.210024 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:24:58.210242 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:24:58.210464 (+ 222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:24:58.210465 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:24:58.210465 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:24:58.210562 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:24:58.210860 (+ 298us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:24:58.210866 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:24:58.210998 (+ 132us) client_negotiation.cc:769] Sending connection context
0809 16:24:58.211084 (+ 86us) client_negotiation.cc:240] Negotiation successful
0809 16:24:58.211194 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":104,"threads_started":1}
I20260809 16:24:58.212535 24472 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.51.189" port: 41049 } has no permanent_uuid. Determining permanent_uuid...
I20260809 16:24:58.214614 24478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:58.205173 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49332 (local address 127.21.51.188:45351)
0809 16:24:58.205360 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0809 16:24:58.205365 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 16:24:58.205424 (+ 59us) server_negotiation.cc:407] Connection header received
0809 16:24:58.205491 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:24:58.205494 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:24:58.205515 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:24:58.205588 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:24:58.206224 (+ 636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:24:58.207241 (+ 1017us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:24:58.208197 (+ 956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:24:58.208563 (+ 366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:24:58.208639 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:24:58.211331 (+ 2692us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:24:58.211337 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:24:58.211338 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:24:58.211344 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:24:58.213437 (+ 2093us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:24:58.213762 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:24:58.213764 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:24:58.213765 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:24:58.213827 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:24:58.213988 (+ 161us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:24:58.213988 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:24:58.213988 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:24:58.214188 (+ 200us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:24:58.214340 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0809 16:24:58.214423 (+ 83us) server_negotiation.cc:299] Negotiation successful
0809 16:24:58.214505 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":28,"server-negotiator.queue_time_us":160,"thread_start_us":107,"threads_started":1}
Aug 09 16:24:58 dist-test-slave-s11t krb5kdc[24112](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292698, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.51.188@KRBTEST.COM for kudu/127.21.51.189@KRBTEST.COM
Aug 09 16:24:58 dist-test-slave-s11t krb5kdc[24112](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292697, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.51.190@KRBTEST.COM
I20260809 16:24:58.220177 24358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:58.212908 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55530 (local address 127.21.51.189:41049)
0809 16:24:58.212975 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0809 16:24:58.212978 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 16:24:58.212988 (+ 10us) server_negotiation.cc:407] Connection header received
0809 16:24:58.213013 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:24:58.213014 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:24:58.213024 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:24:58.213088 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:24:58.213371 (+ 283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:24:58.214030 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:24:58.215018 (+ 988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:24:58.215342 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:24:58.215430 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:24:58.217257 (+ 1827us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:24:58.217258 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:24:58.217259 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:24:58.217265 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:24:58.218966 (+ 1701us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:24:58.219359 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:24:58.219361 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:24:58.219361 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:24:58.219436 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:24:58.219633 (+ 197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:24:58.219635 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:24:58.219635 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:24:58.219766 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:24:58.219854 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0809 16:24:58.220038 (+ 184us) server_negotiation.cc:299] Negotiation successful
0809 16:24:58.220113 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260809 16:24:58.220187 24474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:58.212819 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.189:41049 (local address 127.0.0.1:55530)
0809 16:24:58.212872 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0809 16:24:58.212884 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 16:24:58.212929 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:24:58.213096 (+ 167us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:24:58.213097 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:24:58.213268 (+ 171us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:24:58.213295 (+ 27us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:24:58.213296 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:24:58.214093 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:24:58.214094 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:24:58.214602 (+ 508us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:24:58.214604 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:24:58.215420 (+ 816us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:24:58.215421 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:24:58.215491 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:24:58.215878 (+ 387us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:24:58.215882 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:24:58.217172 (+ 1290us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:24:58.219074 (+ 1902us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:24:58.219075 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:24:58.219075 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:24:58.219249 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:24:58.219522 (+ 273us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:24:58.219522 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:24:58.219523 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:24:58.219571 (+ 48us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:24:58.219875 (+ 304us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:24:58.219876 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:24:58.219926 (+ 50us) client_negotiation.cc:769] Sending connection context
0809 16:24:58.220007 (+ 81us) client_negotiation.cc:240] Negotiation successful
0809 16:24:58.220091 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35}
I20260809 16:24:58.220585 24472 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.51.188" port: 45351 } has no permanent_uuid. Determining permanent_uuid...
I20260809 16:24:58.223677 24231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:58.215443 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55030 (local address 127.21.51.190:45669)
0809 16:24:58.215478 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0809 16:24:58.215480 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:24:58.215488 (+ 8us) server_negotiation.cc:407] Connection header received
0809 16:24:58.215510 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:24:58.215511 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:24:58.215521 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:24:58.215579 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:24:58.216134 (+ 555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:24:58.216821 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:24:58.217536 (+ 715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:24:58.217792 (+ 256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:24:58.217865 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:24:58.220726 (+ 2861us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:24:58.220727 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:24:58.220727 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:24:58.220735 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:24:58.222270 (+ 1535us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:24:58.222631 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:24:58.222632 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:24:58.222633 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:24:58.222704 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:24:58.222903 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:24:58.222904 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:24:58.222904 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:24:58.223020 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:24:58.223196 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0809 16:24:58.223483 (+ 287us) server_negotiation.cc:299] Negotiation successful
0809 16:24:58.223564 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
Aug 09 16:24:58 dist-test-slave-s11t krb5kdc[24112](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292698, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.51.188@KRBTEST.COM for kudu/127.21.51.188@KRBTEST.COM
I20260809 16:24:58.227216 24474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:58.220870 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.188:45351 (local address 127.0.0.1:49346)
0809 16:24:58.220921 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0809 16:24:58.220933 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 16:24:58.220969 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:24:58.221369 (+ 400us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:24:58.221370 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:24:58.221479 (+ 109us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:24:58.221506 (+ 27us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:24:58.221507 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:24:58.222121 (+ 614us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:24:58.222122 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:24:58.222572 (+ 450us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:24:58.222574 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:24:58.222965 (+ 391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:24:58.222966 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:24:58.223021 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:24:58.223349 (+ 328us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:24:58.223352 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:24:58.224631 (+ 1279us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:24:58.226205 (+ 1574us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:24:58.226206 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:24:58.226206 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:24:58.226367 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:24:58.226576 (+ 209us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:24:58.226576 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:24:58.226577 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:24:58.226631 (+ 54us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:24:58.226897 (+ 266us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:24:58.226898 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:24:58.226933 (+ 35us) client_negotiation.cc:769] Sending connection context
0809 16:24:58.227014 (+ 81us) client_negotiation.cc:240] Negotiation successful
0809 16:24:58.227110 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35}
I20260809 16:24:58.227263 24478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:58.220920 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49346 (local address 127.21.51.188:45351)
0809 16:24:58.221251 (+ 331us) server_negotiation.cc:206] Beginning negotiation
0809 16:24:58.221252 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 16:24:58.221260 (+ 8us) server_negotiation.cc:407] Connection header received
0809 16:24:58.221281 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:24:58.221282 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:24:58.221292 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:24:58.221351 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:24:58.221579 (+ 228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:24:58.222039 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:24:58.222650 (+ 611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:24:58.222879 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:24:58.222964 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:24:58.224741 (+ 1777us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:24:58.224743 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:24:58.224743 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:24:58.224749 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:24:58.226104 (+ 1355us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:24:58.226457 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:24:58.226458 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:24:58.226458 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:24:58.226507 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:24:58.226700 (+ 193us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:24:58.226701 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:24:58.226701 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:24:58.226806 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:24:58.226882 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0809 16:24:58.227049 (+ 167us) server_negotiation.cc:299] Negotiation successful
0809 16:24:58.227157 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":44,"server-negotiator.queue_time_us":308}
Aug 09 16:24:58 dist-test-slave-s11t krb5kdc[24112](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292697, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.51.189@KRBTEST.COM
I20260809 16:24:58.229069 24472 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 219add3b418143088dcc3c25683ffa23: Bootstrap starting.
I20260809 16:24:58.230183 24472 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 219add3b418143088dcc3c25683ffa23: Neither blocks nor log segments found. Creating new log.
I20260809 16:24:58.230516 24472 log.cc:824] T 00000000000000000000000000000000 P 219add3b418143088dcc3c25683ffa23: Log is configured to *not* fsync() on all Append() calls
I20260809 16:24:58.231436 24472 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 219add3b418143088dcc3c25683ffa23: No bootstrap required, opened a new log
I20260809 16:24:58.231573 24358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:58.224330 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55540 (local address 127.21.51.189:41049)
0809 16:24:58.224396 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0809 16:24:58.224398 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:24:58.224406 (+ 8us) server_negotiation.cc:407] Connection header received
0809 16:24:58.224431 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:24:58.224432 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:24:58.224440 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:24:58.224494 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:24:58.225049 (+ 555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:24:58.225496 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:24:58.226124 (+ 628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:24:58.226349 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:24:58.226426 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:24:58.228723 (+ 2297us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:24:58.228725 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:24:58.228725 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:24:58.228732 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:24:58.230379 (+ 1647us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:24:58.230711 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:24:58.230712 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:24:58.230712 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:24:58.230773 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:24:58.231020 (+ 247us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:24:58.231021 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:24:58.231021 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:24:58.231171 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:24:58.231356 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0809 16:24:58.231399 (+ 43us) server_negotiation.cc:299] Negotiation successful
0809 16:24:58.231475 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260809 16:24:58.233090 21710 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 16:24:58.233511 24472 raft_consensus.cc:348] T 00000000000000000000000000000000 P 219add3b418143088dcc3c25683ffa23 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d7cb2032f954c669241ffa084b9bda0" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 45669 } } peers { permanent_uuid: "bb957a45319c4018a8fc6cf78f8d668e" member_type: VOTER last_known_addr { host: "127.21.51.189" port: 41049 } } peers { permanent_uuid: "219add3b418143088dcc3c25683ffa23" member_type: VOTER last_known_addr { host: "127.21.51.188" port: 45351 } }
I20260809 16:24:58.233704 24472 raft_consensus.cc:729] T 00000000000000000000000000000000 P 219add3b418143088dcc3c25683ffa23 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 219add3b418143088dcc3c25683ffa23, State: Initialized, Role: FOLLOWER
I20260809 16:24:58.233801 24472 consensus_queue.cc:260] T 00000000000000000000000000000000 P 219add3b418143088dcc3c25683ffa23 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d7cb2032f954c669241ffa084b9bda0" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 45669 } } peers { permanent_uuid: "bb957a45319c4018a8fc6cf78f8d668e" member_type: VOTER last_known_addr { host: "127.21.51.189" port: 41049 } } peers { permanent_uuid: "219add3b418143088dcc3c25683ffa23" member_type: VOTER last_known_addr { host: "127.21.51.188" port: 45351 } }
I20260809 16:24:58.234136 24472 sys_catalog.cc:564] T 00000000000000000000000000000000 P 219add3b418143088dcc3c25683ffa23 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 16:24:58.234122 24481 sys_catalog.cc:455] T 00000000000000000000000000000000 P 219add3b418143088dcc3c25683ffa23 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d7cb2032f954c669241ffa084b9bda0" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 45669 } } peers { permanent_uuid: "bb957a45319c4018a8fc6cf78f8d668e" member_type: VOTER last_known_addr { host: "127.21.51.189" port: 41049 } } peers { permanent_uuid: "219add3b418143088dcc3c25683ffa23" member_type: VOTER last_known_addr { host: "127.21.51.188" port: 45351 } } }
I20260809 16:24:58.234218 24481 sys_catalog.cc:458] T 00000000000000000000000000000000 P 219add3b418143088dcc3c25683ffa23 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 16:24:58.234046 24186 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "6d7cb2032f954c669241ffa084b9bda0"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55030
W20260809 16:24:58.235996 24497 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 219add3b418143088dcc3c25683ffa23: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 16:24:58.236066 24497 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 16:24:58.236371 24362 master_runner.cc:186] Error getting master registration for 127.21.51.190:45669: OK, instance_id { permanent_uuid: "6d7cb2032f954c669241ffa084b9bda0" instance_seqno: 1786292697896430 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 16:24:58.239231 24362 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260809 16:24:58.240116 24225 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.51.188" port: 45351 } attempt: 2
Aug 09 16:24:58 dist-test-slave-s11t krb5kdc[24112](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292697, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.51.190@KRBTEST.COM for kudu/127.21.51.188@KRBTEST.COM
I20260809 16:24:58.247638 24478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:58.240421 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49362 (local address 127.21.51.188:45351)
0809 16:24:58.240476 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0809 16:24:58.240478 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:24:58.240485 (+ 7us) server_negotiation.cc:407] Connection header received
0809 16:24:58.240505 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:24:58.240506 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:24:58.240515 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:24:58.240565 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:24:58.240889 (+ 324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:24:58.241611 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:24:58.242272 (+ 661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:24:58.242476 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:24:58.242549 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:24:58.244586 (+ 2037us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:24:58.244588 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:24:58.244588 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:24:58.244596 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:24:58.246313 (+ 1717us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:24:58.246755 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:24:58.246756 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:24:58.246756 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:24:58.246818 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:24:58.247043 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:24:58.247044 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:24:58.247044 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:24:58.247212 (+ 168us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:24:58.247306 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0809 16:24:58.247468 (+ 162us) server_negotiation.cc:299] Negotiation successful
0809 16:24:58.247550 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260809 16:24:58.247682 24227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:58.240357 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.188:45351 (local address 127.0.0.1:49362)
0809 16:24:58.240408 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0809 16:24:58.240418 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 16:24:58.240456 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:24:58.240567 (+ 111us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:24:58.240568 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:24:58.240763 (+ 195us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:24:58.240809 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:24:58.240810 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:24:58.241685 (+ 875us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:24:58.241686 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:24:58.242187 (+ 501us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:24:58.242188 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:24:58.242547 (+ 359us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:24:58.242548 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:24:58.242605 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:24:58.243070 (+ 465us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:24:58.243076 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:24:58.244486 (+ 1410us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:24:58.246407 (+ 1921us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:24:58.246417 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:24:58.246418 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:24:58.246669 (+ 251us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:24:58.246897 (+ 228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:24:58.246898 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:24:58.246898 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:24:58.246972 (+ 74us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:24:58.247290 (+ 318us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:24:58.247291 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:24:58.247344 (+ 53us) client_negotiation.cc:769] Sending connection context
0809 16:24:58.247433 (+ 89us) client_negotiation.cc:240] Negotiation successful
0809 16:24:58.247548 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34}
I20260809 16:24:58.250589 24225 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6d7cb2032f954c669241ffa084b9bda0: Bootstrap starting.
I20260809 16:24:58.251260 24225 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6d7cb2032f954c669241ffa084b9bda0: Neither blocks nor log segments found. Creating new log.
I20260809 16:24:58.251560 24225 log.cc:824] T 00000000000000000000000000000000 P 6d7cb2032f954c669241ffa084b9bda0: Log is configured to *not* fsync() on all Append() calls
I20260809 16:24:58.252599 24225 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6d7cb2032f954c669241ffa084b9bda0: No bootstrap required, opened a new log
I20260809 16:24:58.254518 24225 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6d7cb2032f954c669241ffa084b9bda0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d7cb2032f954c669241ffa084b9bda0" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 45669 } } peers { permanent_uuid: "bb957a45319c4018a8fc6cf78f8d668e" member_type: VOTER last_known_addr { host: "127.21.51.189" port: 41049 } } peers { permanent_uuid: "219add3b418143088dcc3c25683ffa23" member_type: VOTER last_known_addr { host: "127.21.51.188" port: 45351 } }
I20260809 16:24:58.254633 24225 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6d7cb2032f954c669241ffa084b9bda0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6d7cb2032f954c669241ffa084b9bda0, State: Initialized, Role: FOLLOWER
I20260809 16:24:58.254685 24225 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6d7cb2032f954c669241ffa084b9bda0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d7cb2032f954c669241ffa084b9bda0" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 45669 } } peers { permanent_uuid: "bb957a45319c4018a8fc6cf78f8d668e" member_type: VOTER last_known_addr { host: "127.21.51.189" port: 41049 } } peers { permanent_uuid: "219add3b418143088dcc3c25683ffa23" member_type: VOTER last_known_addr { host: "127.21.51.188" port: 45351 } }
I20260809 16:24:58.254930 24501 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6d7cb2032f954c669241ffa084b9bda0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d7cb2032f954c669241ffa084b9bda0" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 45669 } } peers { permanent_uuid: "bb957a45319c4018a8fc6cf78f8d668e" member_type: VOTER last_known_addr { host: "127.21.51.189" port: 41049 } } peers { permanent_uuid: "219add3b418143088dcc3c25683ffa23" member_type: VOTER last_known_addr { host: "127.21.51.188" port: 45351 } } }
I20260809 16:24:58.254995 24225 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6d7cb2032f954c669241ffa084b9bda0 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 16:24:58.255007 24501 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6d7cb2032f954c669241ffa084b9bda0 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 16:24:58.255133 24186 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6d7cb2032f954c669241ffa084b9bda0 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260809 16:24:58.255213 24186 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6d7cb2032f954c669241ffa084b9bda0 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:24:58.256000 24186 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6d7cb2032f954c669241ffa084b9bda0 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d7cb2032f954c669241ffa084b9bda0" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 45669 } } peers { permanent_uuid: "bb957a45319c4018a8fc6cf78f8d668e" member_type: VOTER last_known_addr { host: "127.21.51.189" port: 41049 } } peers { permanent_uuid: "219add3b418143088dcc3c25683ffa23" member_type: VOTER last_known_addr { host: "127.21.51.188" port: 45351 } }
I20260809 16:24:58.256626 24186 leader_election.cc:288] T 00000000000000000000000000000000 P 6d7cb2032f954c669241ffa084b9bda0 [CANDIDATE]: Term 1 election: Requested vote from peers bb957a45319c4018a8fc6cf78f8d668e (127.21.51.189:41049), 219add3b418143088dcc3c25683ffa23 (127.21.51.188:45351)
I20260809 16:24:58.256851 24431 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "6d7cb2032f954c669241ffa084b9bda0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "219add3b418143088dcc3c25683ffa23"
I20260809 16:24:58.257040 24431 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 219add3b418143088dcc3c25683ffa23 [term 0 FOLLOWER]: Advancing to term 1
W20260809 16:24:58.257818 24517 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 6d7cb2032f954c669241ffa084b9bda0: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 16:24:58.257879 24517 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 16:24:58.258069 24431 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 219add3b418143088dcc3c25683ffa23 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6d7cb2032f954c669241ffa084b9bda0 in term 1.
I20260809 16:24:58.258289 24155 leader_election.cc:302] T 00000000000000000000000000000000 P 6d7cb2032f954c669241ffa084b9bda0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 219add3b418143088dcc3c25683ffa23, 6d7cb2032f954c669241ffa084b9bda0; no voters:
I20260809 16:24:58.258399 24501 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6d7cb2032f954c669241ffa084b9bda0 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:24:58.258584 24501 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6d7cb2032f954c669241ffa084b9bda0 [term 1 LEADER]: Becoming Leader. State: Replica: 6d7cb2032f954c669241ffa084b9bda0, State: Running, Role: LEADER
I20260809 16:24:58.258669 24501 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6d7cb2032f954c669241ffa084b9bda0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d7cb2032f954c669241ffa084b9bda0" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 45669 } } peers { permanent_uuid: "bb957a45319c4018a8fc6cf78f8d668e" member_type: VOTER last_known_addr { host: "127.21.51.189" port: 41049 } } peers { permanent_uuid: "219add3b418143088dcc3c25683ffa23" member_type: VOTER last_known_addr { host: "127.21.51.188" port: 45351 } }
I20260809 16:24:58.259330 24523 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6d7cb2032f954c669241ffa084b9bda0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6d7cb2032f954c669241ffa084b9bda0. Latest consensus state: current_term: 1 leader_uuid: "6d7cb2032f954c669241ffa084b9bda0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d7cb2032f954c669241ffa084b9bda0" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 45669 } } peers { permanent_uuid: "bb957a45319c4018a8fc6cf78f8d668e" member_type: VOTER last_known_addr { host: "127.21.51.189" port: 41049 } } peers { permanent_uuid: "219add3b418143088dcc3c25683ffa23" member_type: VOTER last_known_addr { host: "127.21.51.188" port: 45351 } } }
I20260809 16:24:58.259423 24523 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6d7cb2032f954c669241ffa084b9bda0 [sys.catalog]: This master's current role is: LEADER
I20260809 16:24:58.260468 24529 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 16:24:58.260849 24529 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 16:24:58.260179 24134 master_runner.cc:186] Error getting master registration for 127.21.51.189:41049: OK, instance_id { permanent_uuid: "bb957a45319c4018a8fc6cf78f8d668e" instance_seqno: 1786292698045375 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 16:24:58.262554 24134 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260809 16:24:58.270917 24478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:58.259593 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49378 (local address 127.21.51.188:45351)
0809 16:24:58.259656 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0809 16:24:58.259658 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:24:58.261496 (+ 1838us) server_negotiation.cc:407] Connection header received
0809 16:24:58.261646 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:24:58.261647 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:24:58.261658 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:24:58.261699 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:24:58.263896 (+ 2197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:24:58.264599 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:24:58.266119 (+ 1520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:24:58.266433 (+ 314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:24:58.266491 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:24:58.268053 (+ 1562us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:24:58.268055 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:24:58.268055 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:24:58.268062 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:24:58.269681 (+ 1619us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:24:58.270085 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:24:58.270086 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:24:58.270086 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:24:58.270139 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:24:58.270347 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:24:58.270348 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:24:58.270348 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:24:58.270471 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:24:58.270548 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0809 16:24:58.270750 (+ 202us) server_negotiation.cc:299] Negotiation successful
0809 16:24:58.270826 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260809 16:24:58.272861 24358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:58.259213 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55556 (local address 127.21.51.189:41049)
0809 16:24:58.259253 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0809 16:24:58.259255 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:24:58.259648 (+ 393us) server_negotiation.cc:407] Connection header received
0809 16:24:58.259756 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:24:58.259757 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:24:58.259769 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:24:58.259803 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:24:58.260623 (+ 820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:24:58.261349 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:24:58.262405 (+ 1056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:24:58.263315 (+ 910us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:24:58.263366 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:24:58.265021 (+ 1655us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:24:58.265022 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:24:58.265022 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:24:58.265030 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:24:58.266741 (+ 1711us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:24:58.271420 (+ 4679us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:24:58.271421 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:24:58.271422 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:24:58.271482 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:24:58.272005 (+ 523us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:24:58.272006 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:24:58.272006 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:24:58.272133 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:24:58.272198 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0809 16:24:58.272604 (+ 406us) server_negotiation.cc:299] Negotiation successful
0809 16:24:58.272722 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260809 16:24:58.272902 24431 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 219add3b418143088dcc3c25683ffa23 [term 1 FOLLOWER]: Refusing update from remote peer 6d7cb2032f954c669241ffa084b9bda0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 16:24:58.273206 24523 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 6d7cb2032f954c669241ffa084b9bda0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "219add3b418143088dcc3c25683ffa23" member_type: VOTER last_known_addr { host: "127.21.51.188" port: 45351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 16:24:58.275098 24481 sys_catalog.cc:455] T 00000000000000000000000000000000 P 219add3b418143088dcc3c25683ffa23 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6d7cb2032f954c669241ffa084b9bda0. Latest consensus state: current_term: 1 leader_uuid: "6d7cb2032f954c669241ffa084b9bda0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d7cb2032f954c669241ffa084b9bda0" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 45669 } } peers { permanent_uuid: "bb957a45319c4018a8fc6cf78f8d668e" member_type: VOTER last_known_addr { host: "127.21.51.189" port: 41049 } } peers { permanent_uuid: "219add3b418143088dcc3c25683ffa23" member_type: VOTER last_known_addr { host: "127.21.51.188" port: 45351 } } }
I20260809 16:24:58.275187 24481 sys_catalog.cc:458] T 00000000000000000000000000000000 P 219add3b418143088dcc3c25683ffa23 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 16:24:58.275923 24523 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6d7cb2032f954c669241ffa084b9bda0 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "6d7cb2032f954c669241ffa084b9bda0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d7cb2032f954c669241ffa084b9bda0" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 45669 } } peers { permanent_uuid: "bb957a45319c4018a8fc6cf78f8d668e" member_type: VOTER last_known_addr { host: "127.21.51.189" port: 41049 } } peers { permanent_uuid: "219add3b418143088dcc3c25683ffa23" member_type: VOTER last_known_addr { host: "127.21.51.188" port: 45351 } } }
I20260809 16:24:58.276005 24523 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6d7cb2032f954c669241ffa084b9bda0 [sys.catalog]: This master's current role is: LEADER
I20260809 16:24:58.276467 24481 sys_catalog.cc:455] T 00000000000000000000000000000000 P 219add3b418143088dcc3c25683ffa23 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "6d7cb2032f954c669241ffa084b9bda0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d7cb2032f954c669241ffa084b9bda0" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 45669 } } peers { permanent_uuid: "bb957a45319c4018a8fc6cf78f8d668e" member_type: VOTER last_known_addr { host: "127.21.51.189" port: 41049 } } peers { permanent_uuid: "219add3b418143088dcc3c25683ffa23" member_type: VOTER last_known_addr { host: "127.21.51.188" port: 45351 } } }
I20260809 16:24:58.276554 24481 sys_catalog.cc:458] T 00000000000000000000000000000000 P 219add3b418143088dcc3c25683ffa23 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 16:24:58.276619 24531 mvcc.cc:205] Tried to move back new op lower bound from 7316654892120309760 to 7316654892068253696. Current Snapshot: MvccSnapshot[applied={T|T < 7316654892120309760}]
I20260809 16:24:58.276737 24530 mvcc.cc:205] Tried to move back new op lower bound from 7316654892120309760 to 7316654892068253696. Current Snapshot: MvccSnapshot[applied={T|T < 7316654892120309760}]
I20260809 16:24:58.278478 24231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:58.259036 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55034 (local address 127.21.51.190:45669)
0809 16:24:58.259132 (+ 96us) server_negotiation.cc:206] Beginning negotiation
0809 16:24:58.259134 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:24:58.269047 (+ 9913us) server_negotiation.cc:407] Connection header received
0809 16:24:58.269072 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:24:58.269073 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:24:58.269084 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:24:58.269137 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:24:58.270137 (+ 1000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:24:58.270888 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:24:58.272382 (+ 1494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:24:58.272708 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:24:58.272755 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:24:58.274821 (+ 2066us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:24:58.274823 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:24:58.274823 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:24:58.274831 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:24:58.277027 (+ 2196us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:24:58.277439 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:24:58.277440 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:24:58.277441 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:24:58.277500 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:24:58.277739 (+ 239us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:24:58.277740 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:24:58.277740 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:24:58.277867 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:24:58.277957 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0809 16:24:58.278301 (+ 344us) server_negotiation.cc:299] Negotiation successful
0809 16:24:58.278382 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260809 16:24:58.280179 24354 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.51.188" port: 45351 } attempt: 3
I20260809 16:24:58.283388 24529 catalog_manager.cc:1347] Generated new cluster ID: 5bdc69235ba24bd6a90d6c1ec844f73e
I20260809 16:24:58.283444 24529 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 09 16:24:58 dist-test-slave-s11t krb5kdc[24112](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292698, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.51.189@KRBTEST.COM for kudu/127.21.51.188@KRBTEST.COM
I20260809 16:24:58.289366 24356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:58.280404 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.51.188:45351 (local address 127.0.0.1:49386)
0809 16:24:58.280445 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0809 16:24:58.280454 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 16:24:58.280490 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:24:58.280804 (+ 314us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:24:58.280805 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:24:58.280961 (+ 156us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:24:58.281001 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:24:58.281003 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:24:58.281900 (+ 897us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:24:58.281902 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:24:58.282699 (+ 797us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:24:58.282702 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:24:58.283769 (+ 1067us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:24:58.283770 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:24:58.283829 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:24:58.284307 (+ 478us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:24:58.284312 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:24:58.286480 (+ 2168us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:24:58.288257 (+ 1777us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:24:58.288258 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:24:58.288259 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:24:58.288473 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:24:58.288733 (+ 260us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:24:58.288734 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:24:58.288734 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:24:58.288796 (+ 62us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:24:58.289074 (+ 278us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:24:58.289075 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:24:58.289114 (+ 39us) client_negotiation.cc:769] Sending connection context
0809 16:24:58.289174 (+ 60us) client_negotiation.cc:240] Negotiation successful
0809 16:24:58.289262 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":26}
I20260809 16:24:58.289894 24478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:58.280574 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49386 (local address 127.21.51.188:45351)
0809 16:24:58.280654 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0809 16:24:58.280656 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:24:58.280663 (+ 7us) server_negotiation.cc:407] Connection header received
0809 16:24:58.280705 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:24:58.280706 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:24:58.280715 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:24:58.280763 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:24:58.281085 (+ 322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:24:58.281795 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:24:58.283323 (+ 1528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:24:58.283615 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:24:58.283673 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:24:58.286569 (+ 2896us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:24:58.286570 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:24:58.286570 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:24:58.286576 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:24:58.288166 (+ 1590us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:24:58.288550 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:24:58.288551 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:24:58.288551 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:24:58.288602 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:24:58.288867 (+ 265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:24:58.288868 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:24:58.288868 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:24:58.288992 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:24:58.289049 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0809 16:24:58.289736 (+ 687us) server_negotiation.cc:299] Negotiation successful
0809 16:24:58.289810 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260809 16:24:58.291517 24354 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bb957a45319c4018a8fc6cf78f8d668e: Bootstrap starting.
I20260809 16:24:58.292213 24354 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bb957a45319c4018a8fc6cf78f8d668e: Neither blocks nor log segments found. Creating new log.
I20260809 16:24:58.292502 24354 log.cc:824] T 00000000000000000000000000000000 P bb957a45319c4018a8fc6cf78f8d668e: Log is configured to *not* fsync() on all Append() calls
I20260809 16:24:58.293506 24529 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 16:24:58.294229 24529 catalog_manager.cc:1504] Loading token signing keys...
I20260809 16:24:58.302574 24529 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6d7cb2032f954c669241ffa084b9bda0: Generated new TSK 0
I20260809 16:24:58.302870 24529 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 16:24:58.309432 24354 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bb957a45319c4018a8fc6cf78f8d668e: No bootstrap required, opened a new log
I20260809 16:24:58.311513 24354 raft_consensus.cc:348] T 00000000000000000000000000000000 P bb957a45319c4018a8fc6cf78f8d668e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d7cb2032f954c669241ffa084b9bda0" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 45669 } } peers { permanent_uuid: "bb957a45319c4018a8fc6cf78f8d668e" member_type: VOTER last_known_addr { host: "127.21.51.189" port: 41049 } } peers { permanent_uuid: "219add3b418143088dcc3c25683ffa23" member_type: VOTER last_known_addr { host: "127.21.51.188" port: 45351 } }
I20260809 16:24:58.311646 24354 raft_consensus.cc:729] T 00000000000000000000000000000000 P bb957a45319c4018a8fc6cf78f8d668e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bb957a45319c4018a8fc6cf78f8d668e, State: Initialized, Role: FOLLOWER
I20260809 16:24:58.311717 24354 consensus_queue.cc:260] T 00000000000000000000000000000000 P bb957a45319c4018a8fc6cf78f8d668e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d7cb2032f954c669241ffa084b9bda0" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 45669 } } peers { permanent_uuid: "bb957a45319c4018a8fc6cf78f8d668e" member_type: VOTER last_known_addr { host: "127.21.51.189" port: 41049 } } peers { permanent_uuid: "219add3b418143088dcc3c25683ffa23" member_type: VOTER last_known_addr { host: "127.21.51.188" port: 45351 } }
I20260809 16:24:58.312073 24315 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "6d7cb2032f954c669241ffa084b9bda0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "bb957a45319c4018a8fc6cf78f8d668e"
I20260809 16:24:58.312132 24314 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bb957a45319c4018a8fc6cf78f8d668e [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:24:58.313329 24314 raft_consensus.cc:1261] T 00000000000000000000000000000000 P bb957a45319c4018a8fc6cf78f8d668e [term 1 FOLLOWER]: Refusing update from remote peer 6d7cb2032f954c669241ffa084b9bda0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 16:24:58.313647 24501 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 6d7cb2032f954c669241ffa084b9bda0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bb957a45319c4018a8fc6cf78f8d668e" member_type: VOTER last_known_addr { host: "127.21.51.189" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 16:24:58.314656 24315 raft_consensus.cc:2454] T 00000000000000000000000000000000 P bb957a45319c4018a8fc6cf78f8d668e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6d7cb2032f954c669241ffa084b9bda0 in term 1.
I20260809 16:24:58.312026 24354 sys_catalog.cc:564] T 00000000000000000000000000000000 P bb957a45319c4018a8fc6cf78f8d668e [sys.catalog]: configured and running, proceeding with master startup.
I20260809 16:24:58.312019 24539 sys_catalog.cc:455] T 00000000000000000000000000000000 P bb957a45319c4018a8fc6cf78f8d668e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d7cb2032f954c669241ffa084b9bda0" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 45669 } } peers { permanent_uuid: "bb957a45319c4018a8fc6cf78f8d668e" member_type: VOTER last_known_addr { host: "127.21.51.189" port: 41049 } } peers { permanent_uuid: "219add3b418143088dcc3c25683ffa23" member_type: VOTER last_known_addr { host: "127.21.51.188" port: 45351 } } }
I20260809 16:24:58.318044 24539 sys_catalog.cc:458] T 00000000000000000000000000000000 P bb957a45319c4018a8fc6cf78f8d668e [sys.catalog]: This master's current role is: FOLLOWER
I20260809 16:24:58.318106 24539 sys_catalog.cc:455] T 00000000000000000000000000000000 P bb957a45319c4018a8fc6cf78f8d668e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6d7cb2032f954c669241ffa084b9bda0. Latest consensus state: current_term: 1 leader_uuid: "6d7cb2032f954c669241ffa084b9bda0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d7cb2032f954c669241ffa084b9bda0" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 45669 } } peers { permanent_uuid: "bb957a45319c4018a8fc6cf78f8d668e" member_type: VOTER last_known_addr { host: "127.21.51.189" port: 41049 } } peers { permanent_uuid: "219add3b418143088dcc3c25683ffa23" member_type: VOTER last_known_addr { host: "127.21.51.188" port: 45351 } } }
I20260809 16:24:58.318148 24539 sys_catalog.cc:458] T 00000000000000000000000000000000 P bb957a45319c4018a8fc6cf78f8d668e [sys.catalog]: This master's current role is: FOLLOWER
I20260809 16:24:58.318179 24539 sys_catalog.cc:455] T 00000000000000000000000000000000 P bb957a45319c4018a8fc6cf78f8d668e [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "6d7cb2032f954c669241ffa084b9bda0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d7cb2032f954c669241ffa084b9bda0" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 45669 } } peers { permanent_uuid: "bb957a45319c4018a8fc6cf78f8d668e" member_type: VOTER last_known_addr { host: "127.21.51.189" port: 41049 } } peers { permanent_uuid: "219add3b418143088dcc3c25683ffa23" member_type: VOTER last_known_addr { host: "127.21.51.188" port: 45351 } } }
I20260809 16:24:58.318214 24539 sys_catalog.cc:458] T 00000000000000000000000000000000 P bb957a45319c4018a8fc6cf78f8d668e [sys.catalog]: This master's current role is: FOLLOWER
W20260809 16:24:58.318783 24563 catalog_manager.cc:1558] T 00000000000000000000000000000000 P bb957a45319c4018a8fc6cf78f8d668e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 16:24:58.319044 24563 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 16:24:58.321220 24541 mvcc.cc:205] Tried to move back new op lower bound from 7316654892242903040 to 7316654892068253696. Current Snapshot: MvccSnapshot[applied={T|T < 7316654892120309760}]
W20260809 16:24:58.337268 24358 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:24:58.335922 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55564 (local address 127.21.51.189:41049)
0809 16:24:58.335966 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0809 16:24:58.335968 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:24:58.336570 (+ 602us) server_negotiation.cc:407] Connection header received
0809 16:24:58.336671 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:24:58.336672 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:24:58.336710 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:24:58.336742 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:24:58.337208 (+ 466us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55564: BlockingRecv error: recv got EOF from 127.0.0.1:55564 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
W20260809 16:24:58.338217 24478 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:24:58.336269 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49396 (local address 127.21.51.188:45351)
0809 16:24:58.336306 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0809 16:24:58.336308 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:24:58.337606 (+ 1298us) server_negotiation.cc:407] Connection header received
0809 16:24:58.337775 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:24:58.337776 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:24:58.337786 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:24:58.337815 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:24:58.338160 (+ 345us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49396: BlockingRecv error: recv got EOF from 127.0.0.1:49396 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260809 16:24:58.339305 24231 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:24:58.338656 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55042 (local address 127.21.51.190:45669)
0809 16:24:58.338695 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0809 16:24:58.338697 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:24:58.338805 (+ 108us) server_negotiation.cc:407] Connection header received
0809 16:24:58.338896 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:24:58.338897 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:24:58.338906 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:24:58.338948 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:24:58.339247 (+ 299us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55042: BlockingRecv error: recv got EOF from 127.0.0.1:55042 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
W20260809 16:24:58.341614 24358 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:24:58.340833 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55580 (local address 127.21.51.189:41049)
0809 16:24:58.340866 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0809 16:24:58.340868 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:24:58.341120 (+ 252us) server_negotiation.cc:407] Connection header received
0809 16:24:58.341223 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:24:58.341223 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:24:58.341239 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:24:58.341269 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:24:58.341570 (+ 301us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55580: BlockingRecv error: recv got EOF from 127.0.0.1:55580 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260809 16:24:58.344581 24358 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:24:58.343824 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55596 (local address 127.21.51.189:41049)
0809 16:24:58.343857 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0809 16:24:58.343859 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:24:58.344109 (+ 250us) server_negotiation.cc:407] Connection header received
0809 16:24:58.344200 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:24:58.344201 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:24:58.344210 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:24:58.344239 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:24:58.344539 (+ 300us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55596: BlockingRecv error: recv got EOF from 127.0.0.1:55596 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260809 16:24:58.348409 24358 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:24:58.347677 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55602 (local address 127.21.51.189:41049)
0809 16:24:58.347709 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0809 16:24:58.347710 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 16:24:58.347952 (+ 242us) server_negotiation.cc:407] Connection header received
0809 16:24:58.348046 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:24:58.348047 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:24:58.348056 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:24:58.348084 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:24:58.348370 (+ 286us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55602: BlockingRecv error: recv got EOF from 127.0.0.1:55602 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260809 16:24:58.354188 24358 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:24:58.353448 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55606 (local address 127.21.51.189:41049)
0809 16:24:58.353481 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0809 16:24:58.353483 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:24:58.353710 (+ 227us) server_negotiation.cc:407] Connection header received
0809 16:24:58.353801 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:24:58.353802 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:24:58.353811 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:24:58.353839 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:24:58.354148 (+ 309us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55606: BlockingRecv error: recv got EOF from 127.0.0.1:55606 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260809 16:24:58.364068 24358 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:24:58.363274 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55622 (local address 127.21.51.189:41049)
0809 16:24:58.363308 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0809 16:24:58.363310 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:24:58.363551 (+ 241us) server_negotiation.cc:407] Connection header received
0809 16:24:58.363641 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:24:58.363642 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:24:58.363651 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:24:58.363679 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:24:58.364010 (+ 331us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55622: BlockingRecv error: recv got EOF from 127.0.0.1:55622 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260809 16:24:58.382459 24358 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:24:58.381459 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55634 (local address 127.21.51.189:41049)
0809 16:24:58.381500 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0809 16:24:58.381503 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 16:24:58.381794 (+ 291us) server_negotiation.cc:407] Connection header received
0809 16:24:58.381931 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:24:58.381933 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:24:58.381946 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:24:58.381979 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:24:58.382401 (+ 422us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55634: BlockingRecv error: recv got EOF from 127.0.0.1:55634 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
W20260809 16:24:58.417339 24358 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:24:58.416040 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55640 (local address 127.21.51.189:41049)
0809 16:24:58.416110 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0809 16:24:58.416112 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:24:58.416425 (+ 313us) server_negotiation.cc:407] Connection header received
0809 16:24:58.416537 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:24:58.416538 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:24:58.416551 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:24:58.416582 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:24:58.417276 (+ 694us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55640: BlockingRecv error: recv got EOF from 127.0.0.1:55640 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
W20260809 16:24:58.483810 24358 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:24:58.482915 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55646 (local address 127.21.51.189:41049)
0809 16:24:58.482974 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0809 16:24:58.482976 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:24:58.483116 (+ 140us) server_negotiation.cc:407] Connection header received
0809 16:24:58.483204 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:24:58.483205 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:24:58.483217 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:24:58.483279 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:24:58.483739 (+ 460us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55646: BlockingRecv error: recv got EOF from 127.0.0.1:55646 (error 108)
Metrics: {"server-negotiator.queue_time_us":30}
W20260809 16:24:58.613987 24358 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:24:58.613093 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55660 (local address 127.21.51.189:41049)
0809 16:24:58.613178 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0809 16:24:58.613180 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:24:58.613296 (+ 116us) server_negotiation.cc:407] Connection header received
0809 16:24:58.613399 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:24:58.613401 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:24:58.613413 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:24:58.613493 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:24:58.613916 (+ 423us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55660: BlockingRecv error: recv got EOF from 127.0.0.1:55660 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
I20260809 16:24:58.738039 24501 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6d7cb2032f954c669241ffa084b9bda0 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "6d7cb2032f954c669241ffa084b9bda0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d7cb2032f954c669241ffa084b9bda0" member_type: VOTER last_known_addr { host: "127.21.51.190" port: 45669 } } peers { permanent_uuid: "bb957a45319c4018a8fc6cf78f8d668e" member_type: VOTER last_known_addr { host: "127.21.51.189" port: 41049 } } peers { permanent_uuid: "219add3b418143088dcc3c25683ffa23" member_type: VOTER last_known_addr { host: "127.21.51.188" port: 45351 } } }
I20260809 16:24:58.738232 24501 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6d7cb2032f954c669241ffa084b9bda0 [sys.catalog]: This master's current role is: LEADER
W20260809 16:24:58.871896 24358 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:24:58.871259 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55666 (local address 127.21.51.189:41049)
0809 16:24:58.871335 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0809 16:24:58.871337 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:24:58.871346 (+ 9us) server_negotiation.cc:407] Connection header received
0809 16:24:58.871373 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:24:58.871374 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:24:58.871384 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:24:58.871435 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:24:58.871799 (+ 364us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55666: BlockingRecv error: recv got EOF from 127.0.0.1:55666 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
I20260809 16:24:59.236519 24497 catalog_manager.cc:1259] Loaded cluster ID: 5bdc69235ba24bd6a90d6c1ec844f73e
I20260809 16:24:59.236593 24497 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 219add3b418143088dcc3c25683ffa23: loading cluster ID for follower catalog manager: success
I20260809 16:24:59.237443 24497 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 219add3b418143088dcc3c25683ffa23: acquiring CA information for follower catalog manager: success
I20260809 16:24:59.237764 24497 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 219add3b418143088dcc3c25683ffa23: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 16:24:59.319545 24563 catalog_manager.cc:1259] Loaded cluster ID: 5bdc69235ba24bd6a90d6c1ec844f73e
I20260809 16:24:59.319644 24563 catalog_manager.cc:1552] T 00000000000000000000000000000000 P bb957a45319c4018a8fc6cf78f8d668e: loading cluster ID for follower catalog manager: success
I20260809 16:24:59.320477 24563 catalog_manager.cc:1574] T 00000000000000000000000000000000 P bb957a45319c4018a8fc6cf78f8d668e: acquiring CA information for follower catalog manager: success
I20260809 16:24:59.320858 24563 catalog_manager.cc:1602] T 00000000000000000000000000000000 P bb957a45319c4018a8fc6cf78f8d668e: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 16:24:59.387774 24670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:59.385268 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55672 (local address 127.21.51.189:41049)
0809 16:24:59.385421 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0809 16:24:59.385423 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:24:59.385440 (+ 17us) server_negotiation.cc:407] Connection header received
0809 16:24:59.385503 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:24:59.385504 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:24:59.385557 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:24:59.385633 (+ 76us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 16:24:59.385788 (+ 155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:24:59.386292 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:24:59.387037 (+ 745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:24:59.387225 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:24:59.387276 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:24:59.387429 (+ 153us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 16:24:59.387538 (+ 109us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 16:24:59.387600 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0809 16:24:59.387635 (+ 35us) server_negotiation.cc:299] Negotiation successful
0809 16:24:59.387653 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95,"thread_start_us":60,"threads_started":1}
I20260809 16:24:59.392375 24679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:59.389743 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49408 (local address 127.21.51.188:45351)
0809 16:24:59.389969 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0809 16:24:59.389971 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:24:59.389980 (+ 9us) server_negotiation.cc:407] Connection header received
0809 16:24:59.390030 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:24:59.390031 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:24:59.390081 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:24:59.390165 (+ 84us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 16:24:59.390325 (+ 160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:24:59.390809 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:24:59.391549 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:24:59.391762 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:24:59.391839 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:24:59.392032 (+ 193us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 16:24:59.392124 (+ 92us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 16:24:59.392205 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0809 16:24:59.392258 (+ 53us) server_negotiation.cc:299] Negotiation successful
0809 16:24:59.392280 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":201,"thread_start_us":49,"threads_started":1}
I20260809 16:24:59.396844 24670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:59.394015 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55678 (local address 127.21.51.189:41049)
0809 16:24:59.394084 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0809 16:24:59.394086 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:24:59.394194 (+ 108us) server_negotiation.cc:407] Connection header received
0809 16:24:59.394271 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:24:59.394272 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:24:59.394283 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:24:59.394334 (+ 51us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 16:24:59.394521 (+ 187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:24:59.395268 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:24:59.396112 (+ 844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:24:59.396320 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:24:59.396409 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:24:59.396538 (+ 129us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 16:24:59.396572 (+ 34us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 16:24:59.396632 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0809 16:24:59.396716 (+ 84us) server_negotiation.cc:299] Negotiation successful
0809 16:24:59.396743 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260809 16:24:59.397414 24679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:59.394275 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49412 (local address 127.21.51.188:45351)
0809 16:24:59.394319 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0809 16:24:59.394321 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:24:59.394498 (+ 177us) server_negotiation.cc:407] Connection header received
0809 16:24:59.394608 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:24:59.394609 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:24:59.394620 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:24:59.394671 (+ 51us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 16:24:59.394820 (+ 149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:24:59.395513 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:24:59.396486 (+ 973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:24:59.396814 (+ 328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:24:59.396878 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:24:59.397147 (+ 269us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 16:24:59.397175 (+ 28us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 16:24:59.397205 (+ 30us) server_negotiation.cc:1035] Waiting for connection context
0809 16:24:59.397328 (+ 123us) server_negotiation.cc:299] Negotiation successful
0809 16:24:59.397355 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260809 16:24:59.397472 24689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:24:59.393983 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55054 (local address 127.21.51.190:45669)
0809 16:24:59.394084 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0809 16:24:59.394086 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:24:59.394149 (+ 63us) server_negotiation.cc:407] Connection header received
0809 16:24:59.394249 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:24:59.394250 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:24:59.394300 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:24:59.394380 (+ 80us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 16:24:59.394576 (+ 196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:24:59.395237 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:24:59.396313 (+ 1076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:24:59.396619 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:24:59.396677 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:24:59.397101 (+ 424us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 16:24:59.397197 (+ 96us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 16:24:59.397258 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0809 16:24:59.397362 (+ 104us) server_negotiation.cc:299] Negotiation successful
0809 16:24:59.397389 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80,"thread_start_us":55,"threads_started":1}
I20260809 16:24:59.399061 21710 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrNqUCn/build/release/bin/kudu with pid 24134
I20260809 16:24:59.409045 21710 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrNqUCn/build/release/bin/kudu with pid 24238
I20260809 16:24:59.419035 21710 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrNqUCn/build/release/bin/kudu with pid 24362
2026-08-09T16:24:59Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (3005 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (6036 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (281708 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST